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ABSTRACT 


Test  and  Evaluation  (T&E)  is  an  integral  part  of  every  acquisition  program,  as 
such,  it  consumes  considerable  program  resources.  The  Department  of  Defense  (DoD) 
T&E  program  management  requirements  arc  written  to  meet  the  risk  reduction  needs  of 
large  acquisition  programs,  but  do  not  provide  the  details  needed  to  consistently  scale 
T&E  management  efforts  for  smaller  programs  across  DoD.  This  research  study 
investigates  ways  that  the  T&E  burden  to  programs  differs  based  upon  the  Acquisition 
Category  (AC  AT),  application  of  T&E  oversight,  and  commodity  area.  The  primary  data 
source  for  the  study  was  each  program’s  Test  and  Evaluation  Master  Plan  (TEMP).  The 
majority  of  the  programs  evaluated  are  part  of  the  DoD  Chemical  Biological  Defense 
Program  (CBDP).  This  research  effort  found  ACAT-based  patterns  in  requirements 
definitions  and  ACAT-induccd  variability  in  test  site  usage.  The  study  also  determined 
that  the  TEMP  documentation  burden  was  similar  across  all  ACATs,  and  consistently 
higher  than  Service  guidance.  Future  studies  using  this  analysis  methodology  arc 
required  to  determine  whether  the  T&E  burden  to  CBDP  programs  is  typical  or  unique 
within  DoD. 
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1. 


INTRODUCTION 


A.  BACKGROUND 

The  Department  of  Defense  (DoD)  and  individual  Service  (Army,  Navy,  Air 
Force,  and  Marine  Corps)  guidance  for  acquisition  system  testing  focuses  primarily  on 
large  Acquisition  Category  I  (ACAT  1)  programs.  Therefore,  managers  of  medium, 
AC  AT  11,  and  small,  ACAT  III,  DoD  programs  must  work  closely  with  the  Test  and 
Evaluation  (T&E)  community  to  effectively  tailor  both  test  documentation  and  test  scope 
to  balance  the  test  burden  with  other  program  requirements.  A  Program,  Project,  or 
Product  Manager  (PM)  is  required  to  execute  their  acquisition  program  within  the  cost, 
schedule,  and  technical  constraints  of  the  program  baseline.  The  rest  of  the  acquisition 
community,  including  the  T&E  organizations,  is  not  constrained  by  these  same  overall 
goals. 

Anecdotal  PM  Office  (PMO)  experiences  point  to  historical  T&E  community 
inflexibility,  especially  in  the  tailoring  of  T&E  documentation  scope  based  upon  program 
size  and  complexity.  While  some  programs  require  specialized  testing,  such  as 
laboratory  chemical  or  biological  agent  testing,  most  general  types  of  program  tests  arc 
required  across-the-board.  While  many  of  these  tests  arc  applicable  to  the  majority  of 
weapon  systems,  the  scope  of  T&E  and  the  associated  documentation  should  still  be 
tailored  to  strike  a  balance  between  user  needs  and  overall  program  constraints.  PMO 
experiences  also  point  to  a  proclivity  to  document  and  test  more,  rather  than  less, 
whenever  a  choice  must  be  made  by  the  T&E  activities.  Requiring  additional  tests  or 
more  detailed  documentation  is  unlikely  to  cause  any  personal  or  organizational  issues 
for  a  tester,  while  agreeing  to  less  of  each  will  likely  raise  thorny  questions. 

The  general  PMO  sense  that  the  T&E  burden,  driven  by  the  technical  nature  of 
the  requirements  for  smaller  systems,  standardized  T&E  documentation  without  regard 
for  program  scope,  and  the  lack  of  program-tailored  test  efforts  were  the  issues  that 
sparked  this  research  study.  The  three  research  questions  defined  below  were  chosen 
specifically  to  investigate  whether  these  PMO-held  notions  regarding  T&E  are  justified. 

I 


B. 


RESEARCH  QUESTIONS 


1.  Primary  Research  Question 

•  Is  the  T&E  Level  of  Effort  (LOE)  proportional  to  a  program’s  designated 
ACAT  level? 

2.  Secondary  Research  Questions 

•  How  docs  program  oversight  by  the  Director  Operational  Test  and 

Evaluation  (DOT&E)  or  the  Army  drive  the  T&E  LOE? 

•  Arc  ChemicaL'Biological  (C/B)  programs  a  special  ease  for  T&E  LOE? 

C.  SCOPE  AND  METHODOLOGY 

1.  Scope 

This  research  study  will  evaluate  the  T&E  burden  for  acquisition  programs,  based 
primarily  on  the  data  available  in  each  program’s  TEMP.  The  study  focuses  on  the 
CBDP  within  DoD.  but  also  includes  data  for  two  ACAT  I  non-CBDP  programs.  A 
detailed  analysis  is  performed  to  determine  differences  in  T&E  requirements, 
documentation  burden,  and  other  related  factors  that  impact  the  scope  of  T&E  for 
acquisition  programs. 

This  study  is  bounded  by  the  following  conditions: 

•  Only  programs  with  a  TEMP  containing  sufficient  detail  for  a  comparative 
analysis  were  included  in  this  study. 

•  Software-based  programs,  such  as  Information  Technology  (IT)  and 
Command.  Control,  Communications,  Computers,  Intelligence, 
Surveillance  and  Reconnaissance  (C4ISR)  were  not  evaluated. 

•  The  inconsistent  nature  of  program  cost  summaries  between  TEMPs,  and 
as  reported  in  DoD  budget  materials,  precluded  an  analysis  of  costs,  either 
at  the  program-level  or  for  T&E  efforts. 

2.  Methodology 

A  summary  of  the  approach  used  to  conduct  this  research  project  is: 
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•  Conduct  a  literature  search,  of  both  the  publicly-accessiblc  and  the  For 
Official  Use  Only  (FOUO)  sections,  of  the  Joint  Program  Executive 
Office  for  Chemical  Biological  Defense  (JPEO-CBD)  program  database  to 
acquire  CBDP-spccific  acquisition  program  documents. 

•  Gather  publicly-available  funding  data  on  CBDP  programs  from  the  DoD 
budget  materials  contained  on  the  Office  of  the  Under  Secretary  of 
Defense,  Comptroller  (OUSD(C))  web  site  to  supplement  program- 
specific  acquisition  documents. 

•  Investigate  the  availability  and  usefulness  of  detailed  T&E  documents, 
such  as  Detailed  Test  Plans  (DTPs)  and  Event  Design  Plans  (EDPs),  and 
gather  information  on  tests  conducted  by  the  Army  Test  and  Evaluation 
Command  (ATEC)  using  their  ATEC  Decision  Support  System  (ADSS). 

•  Perform  electronic  searches  to  identify  reports,  theses,  congressional 
testimony,  and  periodical  articles  related  to  T&E  and  the  CBDP. 

•  Review  the  collected  information  to  determine  whether  or  not  each  data 
type  has  the  potential  to  contribute  to  research  question  answers. 

•  Determine  the  core  set  of  documents  required  for  the  analysis. 

•  Request  copies  of  core  documents  for  non-CBDP  programs  to  support  a 
comparative  analysis  of  T&E  LoE  for  CBDP  and  non-CBDP  programs. 

•  Extract  all  potentially  useful  data  elements  from  several  documents  and 
perform  a  preliminary  analysis  to  determine  what,  if  any,  top-level  or 
derived  relationships  exist  within  the  data  that  can  contribute  to  research 
question  answers. 

•  Finalize  the  data  analysis  approach  and  perform  the  analysis  based  on  the 
relationships  identified  above. 

•  Formulate  conclusions  and  recommendations  based  upon  the  results  of  the 
analysis,  taking  the  study's  limitations  into  account.  Provide  suggestions 
for  future  related  study. 

D.  ORGANIZATION  OF  STUDY 

This  Joint  Applied  Project  (JAP)  contains  the  information  necessary  to  understand 
this  research  effort;  starting  with  this  introduction,  followed  by  a  background  section,  a 
data  summary,  data  analysis,  and  conclusions  and  recommendations.  Additional 
infonnation  is  included  at  the  end  of  this  project  paper  to  define  the  acronyms  used. 
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provide  facts  on  the  programs  investigated,  list  all  references  used,  and  to  define  the 
initial  distribution  for  the  document.  Additional  details  for  each  major  chapter  in  this 
document  arc  listed  below. 

Chapter  I:  Introduction.  This  chapter  describes  the  rationale  behind  performing 
this  research  study,  defines  the  research  questions  guiding  the  effort,  defines  the  scope 
and  methodology  used,  defines  the  JAP  organizational  structure,  and  lists  several 
beneficial  outputs  of  this  study. 

Chapter  11:  Background.  Chapter  II  provides  the  basic  knowledge  needed  to 
understand  the  data,  analysis,  and  conclusions  presented  in  this  JAP.  It  begins  with  an 
introduction  to  the  acquisition  process,  including  definitions  of  the  acquisition  categories 
used  by  DoD  to  determine  management  processes.  Since  all  but  three  of  the  programs 
studied  fall  within  the  CBDP,  the  history  and  organizational  structure  of  the  CBDP 
follows.  The  chapter  concludes  with  an  overview  of  T&E,  presented  in  four  parts;  the 
basis  for  T&E  and  the  TEMP,  test  types,  program  oversight,  and  a  review  of  the  T&E 
organizations  used  by  the  programs  in  this  study. 

Chapter  III:  Data  Summary.  The  data  collected  and  analyzed  in  this  JAP  is 
presented  in  this  chapter,  along  with  a  discussion  that  characterizes  all  of  the  data 
collected  during  this  research  The  data  is  partitioned  into  three  categories:  principle 
data,  which  is  the  primary  information  source  for  analysis  in  Chapter  IV;  secondary  data, 
which  only  indirectly  supports  answers  to  the  research  questions;  and  unavailable, 
inconsistent,  and/or  deficient  data,  which  encompasses  information  that  cither  was  not 
available  or  could  not  effectively  be  analyzed. 

Chapter  IV:  Data  Analysis.  This  chapter  analyzes  the  data  presented  in  Chapter 
111.  and  presents  the  results  obtained.  The  limitations  and  scope  of  the  analysis  arc 
discussed  to  define  the  limits  of  the  results,  develops  a  thrcc-pronged  approach  to 
execution,  and  then  uses  this  approach  to  analyze  the  data.  The  three  focus  areas  for  the 
analysis  arc:  system  requirements,  documentation,  and  test  management  scope. 
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Chapter  V:  Conclusions  and  Recommendations.  The  results  of  the  analysis 
presented  in  Chapter  IV  arc  rolled-up  into  overall  conclusions  and  recommendations  in 
this  chapter.  The  answers  to  the  research  questions  defined  in  Chapter  II  arc  presented 
with  supporting  rationale.  Recommendations  for  future  academic  study  and  for  T&E 
program  implementation  within  DoD  arc  also  presented.  Finally,  an  overall  summary  of 
the  research  effort  is  presented. 

E.  BENEFITS  OF  THE  STUDY 

A  PM's  ability  to  balance  all  aspects  of  an  acquisition  program  to  meet  cost, 
schedule,  and  performance  requirements  depends  upon  many  factors.  T&E  is  both  an 
essential  risk  reduction  element  and  a  large  programmatic  burden.  Balancing  program 
T&E  efforts,  in  accordance  with  program  size,  system  complexity,  budget,  and  schedule 
is  critical  to  the  successful  acquisition  of  new  capabilities  for  the  war-fighter. 

DoD  provides  T&E  guidance,  targeted  primarily  for  ACAT  I  programs.  This 
study  investigates  whether  program  requirements,  documentation,  and  test  scope  arc 
tailored  due  to  a  program’s  ACAT,  OSD/DOT&E  oversight  status,  and  materiel 
commodity  area.  The  key  outputs  of  this  study  are:  a  methodology  that  can  be  applied 
DoD-wide  to  determine  technical  versus  operational  requirements  drivers  for  T&E,  the 
characterization  of  the  TEMP  documentation  burden  by  ACAT,  insights  into  the  effects 
of  oversight  on  ACAT  III  programs,  and  -  with  the  help  of  future  research  efforts  -  the 
ability  to  determine  whether  the  T&E  burden  for  CBDP  efforts  is  unique  or  typical  within 
DoD. 
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II.  BACKGROUND 


Galileo  Galilei  said,  “All  truths  are  easy  to  understand  onee  they  arc  discovered; 
the  point  is  to  discover  them."  [1]  While  the  discovery  of  ‘truths'  is  the  goal  of  any 
research  study,  the  task  is  nearly  impossible  without  first  gaining  a  basic  understanding  of 
the  area  of  interest.  Therefore,  this  chapter  provides  the  background  information  needed 
to  understand  the  area  of  this  research  effort,  which  is  Test  and  Evaluation  (T&E)  within 
the  framework  of  the  Department  of  Defense  (DoD)  acquisition  process. 

This  chapter  begins  with  a  top-level  introduction  to  the  DoD  acquisition  process 
and  program  acquisition  category  designators,  followed  by  an  overview  of  the  Chemical 
and  Biological  Defense  Program  (CBDP)  -  which  is  the  primary  domain  of  the  programs 
studied.  Finally,  the  chapter  discusses  the  rationale  for  T&E,  the  types  of  DoD  T&E 
efforts,  introduces  program  oversight,  and  provides  a  summary  of  the  T&E  organizations 
reviewed. 

A.  DOD  ACQUISITION  OVERVIEW 

1.  The  Acquisition  Life-Cycle  Process  and  T&E 

The  acquisition  of  new  products  within  DoD  has  evolved  significantly  over  the 
past  few  decades.  Major  changes  include  the  addition  of  the  requirement  for  independent 
operational  test  agencies  within  each  Service;  the  creation  of  the  Congressional  oversight 
organization,  the  Director  of  Operational  Test  and  Evaluation  (DOT&E);  and  the 
sweeping  acquisition  reform  initiatives  implemented  in  1994  and  1995  by  the  Secretary 
of  Defense,  Dr.  William  Perry.  Many  of  these  changes  were  targeted  at  improving  the 
T&E  aspects  of  the  acquisition  process. 

An  overview  of  the  current  DoD  acquisition  life  cycle  is  depicted  in  Figure  1. 
The  acquisition  of  new  products  for  DoD  has  proven  to  be  a  lengthy  and  inherently  risky 
management  challenge.  Tins  multi-phased  life  cycle  approach  is  designed  to  focus 
acquisition  efforts  to  reduce  risk,  with  T&E  a  key  risk  reduction  tool.  The  life  cycle 
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starts  with  Concept  Refinement  (CR)  and  Technology  Development  (TD)  before  formal 
system  acquisition  begins,  proceeds  to  formal  acquisition  with  System  Development  and 
Demonstration  (SDD)  and  Low  Rate  Initial  Production  (LRIP)  followed  by  Full  Rate 
Production  (FRP).  and  completes  with  the  Operations  and  Support  (O&S)  phase.  A  more 
detailed  discussion  of  each  phase  and  the  top-level  T&E  elements  in  each  follows.  A 
brief  discussion  of  each  test  type  will  be  presented  in  Section  A. 2  of  this  chapter. 


Figure  1.  DoD  Acquisition  Life  Cycle.  (After:  [2,  p.  2-2]) 


During  the  CR  phase,  a  conceptual  technology  approach  is  developed,  based  on 
an  analysis  of  available  technology  alternatives.  The  Joint  Requirements  Oversight 
Council  (JROC)-validated  Initial  Capabilities  Document  (1CD)  is  the  requirements 
document  that  guides  these  efforts.  Analysis  of  existing  data  in  order  to  develop  a  viable 
approach  is  predominant  in  this  early  phase.  Some  Modeling  and  Simulation  (M&S)  and 
limited  laboratory  testing  to  prove  the  viability  of  the  concept  selected  may  also  occur. 


Milestone  A  (MS  A)  marks  the  end  of  the  CR  phase,  and  is  the  decision  point  to 
determine  whether  to  proceed  into  TD.  The  first  overarching  T&E  document  for  the 
program,  the  Test/Evaluation  Strategy  (TES),  is  also  completed  by  the  end  of  CR. 

In  the  TD  phase,  the  viability  of  the  technology  approach  selected  is  proven. 
Prototype  subsystems  and  components  arc  developed  and  technically  tested,  and  more 
targeted  M&S  and  laboratory  testing  may  also  be  conducted  to  determine  the  limits  and 
maturity  of  the  chosen  technologies.  By  the  end  of  the  TD  phase,  several  key  documents 
arc  formalized,  including  the  first  version  of  the  program  TEMP,  the  JROC-validated 
Capability  Development  Document  (CDD),  and  an  Early  Operational  Assessment  (EOA) 
which  assesses  the  operational  impacts  of  the  technology  approach.  Milestone  B  (MS  B) 
ends  this  phase  and  is  the  decision  point  for  the  initiation  of  a  new  acquisition  program. 
In  addition,  MS  B  marks  the  closure  of  the  ability  of  programs  to  request  cither  a  partial 
or  a  full  waiver  from  Live  Fire  T&E  (LFT&E). 

After  a  successful  MS  B  decision,  the  acquisition  program  officially  enters  the 
SDD  phase.  A  Program  Manager  (PM)  is  assigned  and  delegated,  through  a  formal 
charter,  cost,  schedule,  and  performance  responsibility  for  the  program.  SDD  is  broken 
into  two  sub-phases,  system  integration  and  system  demonstration,  with  the  Design 
Readiness  Review  (DRR)  as  the  dividing  gate.  Prior  to  the  DRR.  the  prototype  and 
subsystem  development  efforts  that  began  in  TD.  continue,  with  the  goal  to  develop  fully 
functional  elements  for  incorporation  into  working  system  Engineering  Development 
Models  (EDMs).  After  the  DRR.  the  EDMs  are  tested  to  determine  the  design’s  ability  to 
meet  the  CDD  requirements.  In  addition  to  technical  tests,  one  or  more  Operational 
Assessments  (OA)  are  performed  during  SDD  to  determine  whether  the  system  has  the 
potential  to  be  both  operationally  effective  and  suitable.  At  the  end  of  SDD.  the  TEMP 
is  updated  and  the  JROC-validated  Capability  Production  Document  (CPD)  is  completed. 
Milestone  C  (MS  C)  completes  this  acquisition  phase. 

A  program  enters  Production  and  Deployment  (PD)  and  begins  LRIP  after  MS  C. 
The  initial  LRIP  systems  arc  primarily  used  for  T&E,  which  includes  10T&E  for  singlc- 
Scrvicc  programs  or  MOT&E  for  joint  programs,  system-level  LFT&E,  and  technical 
tests  to  verify  production  fixes  for  any  technical  issues  found  with  the  engineering 

9 


development  models  from  SDD.  Later,  LR1P  systems  are  also  typically  fielded  after 
successful  completion  of  testing  in  order  to  provide  the  system’s  Initial  Operational 
Capability  (IOC).  The  TEMP  is  updated  again  for  the  Full  Rate  Production  Decision 
Review  (FRPDR),  to  include  any  additional  testing  required  on  FRP  systems,  such  as 
Follow-On  T&E  (FOT&E),  as  a  result  of  tests  on  the  LR1P  systems. 

The  dividing  line  between  FRP  and  the  final  phase  of  acquisition.  Operations  and 
Support  is  not  as  clear-cut  as  with  the  previously  discussed  boundaries.  This  is  due  to  the 
overlap  of  system  support  for  early-fielded  units  and  the  production  and  deployment  of 
systems  to  units  with  later  fielding  dates.  FOT&E  is  conducted  using  FRP  systems,  if 
required,  during  this  phase.  Once  all  planned  systems  have  been  fielded,  the  Full 
Operational  Capability  (FOC)  has  been  achieved.  O&S,  or  sustainment,  activities  also 
include  any  tests  required  due  to  system  problems  identified  after  fielding,  user  support, 
and  system  disposal  activities. 

2.  DoD  Acquisition  Categories  (ACAT) 

The  Department  of  Defense  organizes  acquisition  programs  into  three  Acquisition 
Categories  (ACATs):  ACAT  I.  ACAT  11,  ACAT  Ill.  The  ACAT  assigned  to  a  program  is 
primarily  based  on  the  level  of  funding  required  to  execute  the  development  and 
production  program  activities,  however  program  complexity  and  risk  arc  other  factors 
that  can  also  drive  the  final  ACAT  determination.  The  ACAT  assigned  to  a  program  also 
determines  the  types  and  level  of  reviews  and  the  assigned  Milestone  Decision  Authority 
(MDA),  with  some  requirements  determined  by  statute.  DoD  also  executes  non-standard 
development  programs,  such  as  Advanced  Concept  Technology  Demonstrations 
(ACTDs),  Advanced  Technology  Demonstrations  (ATDs),  and  Joint  Warfighting 
Experiments  (JWEs).  A  summary  of  the  characteristics  of  each  ACAT,  including 
funding  criteria,  milestone  review  forum,  and  MDA  is  shown  in  Figure  2. 
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Table  3-1 

Categories  of  acquisition  programs  and  milestone  decision  authorities 
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Figure  2.  ACAT  Definitions.  (From:  |3J) 


An  ACAT  I  is  normally  considered  a  Major  Defense  Acquisition  Program 
(MDAP),  and  encompasses  the  largest  defense  programs.  ACAT  1  programs  are  further 
divided  according  to  specific  oversight  requirements  and  the  type  of  program.  ACAT  1A 
and  ACAT  1AM  designations  are  for  Major  Automated  Information  System  (MA1S) 
programs.  ACAT  ID  and  1C  programs  differ  primarily  in  the  milestone  review  forum  and 
the  MDA  assigned  to  the  program. 


ACAT)  II  programs  arc  acquisition  programs  that  do  not  meet  the  criteria  for  an 
ACAT  I  program,  but  do  meet  the  criteria  for  a  major  system,  or  have  been  designated  as 
a  “special  interest'  program  by  the  MDA. 


ACAT  Ill  programs  arc  non-major  programs  that  have  the  MDA  designated  by 
the  AAE  or  CIO.  These  programs  use  In-Process  Reviews  (IPRs)  as  the  milestone 
review  forum. 


The  majority  of  programs  within  the  CBDP  are  ACAT  Ill-level,  with  the  larger 
programs  designated  as  ACAT  II.  Other  ACAT  I-D  programs,  such  as  the  Stryker 


Nuclear,  Biological,  Chemical,  Reconnaissance  Vehicle  (NBCRV),  incorporate  CBDP 
mission  packages,  but  are  still  managed  by  non-CBDP  organizations  with  JPEO/JPM 
support. 

B.  THE  CHEMICAL  AND  BIOLOGICAL  DEFENSE  PROGRAM 

The  overwhelming  majority  of  programs  evaluated  for  this  project,  20  of  23,  arc 
managed  as  part  of  the  Chemical  and  Biological  Defense  Program  within  DoD. 
Additionally,  one  of  the  other  three  programs  includes  a  fully-integrated  CBDP-managcd 
mission  equipment  package.  Therefore,  a  basic  knowledge  of  the  CBDP  is  essential  to 
understanding  all  aspects  of  this  report.  This  section  outlines  the  history  of  the  CBDP. 
with  a  focus  on  organizational  structure,  defines  the  commodity  areas  within  the  CBDP. 
and  describes  the  unique  aspects  of  the  CBDP  T&E  management  structure. 

The  CBDP  was  established  in  1994  to  provide  for  centralized  management  and 
control  of  all  DoD  Chemical  and  Biological  (CB)  defense  programs.  This  action  was  in 
response  to  Public  Law  No.  103-160,  Section  1701  (50  USC  1522)  which  “mandates  the 
coordination  and  integration  of  all  Department  of  Defense  CB  defense  programs."  [4] 
Deficiencies  in  DoD's  CB  defense  capabilities  identified  during  Operation  Desert  Storm 
(ODS)  provided  much  of  the  impetus  for  Congress  to  mandate  improvements  in  CB 
defense  management. 

The  CBDP  was  initiated  using  the  management  structure  shown  in  Figure  2.  This 
organizational  scheme  provided  DoD-levcl  oversight  of  service-specific  CB  program 
offices.  Within  this  structure,  the  Joint  Service  Integration  Group  (JSIG)  was  tasked  with 
oversight  and  guidance  on  training,  doctrine,  and  requirements  to  ensure  that  service- 
spccific  and  warfighting  Combatant  Commanders  (COCOMs)  requirements  and  priorities 
for  acquisition  were  met  The  Joint  Service  Materiel  Group  (JSMG)  was  responsible  for 
the  CB  technology,  sustainment,  and  research  and  development  of  CB  items  to  mitigate 
program-wide  risk  and  to  minimize  duplication  of  efforts  across  commodity  areas.  The 
JSIG  and  JSMG  were  under  to  the  Joint  Nuclear  Biological  Chemical  (NBC)  Defense 
Board  (JNBCDB)  which  reported  to  the  NBCD  Steering  Committee.  Additionally,  the 
Joint  Program  Office  for  Biological  Defense  (JPO-BD)  was  linked  to  both  the  JNBCDB 
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and  the  medical  program  within  DoD.  Figure  3,  extracted  from  page  23  of  the  General 
Accounting  Office  (GAO)  report  CAO/NSIAD-99-159,  shows  a  slightly  different  view 
of  the  CBDP.  with  the  inclusion  of  the  Soldier  and  Biological  Chemical  Command 
(SBCCOM)  as  a  complimentary  material  development  organization  to  the  JPO-  BD. 

The  CBDP  commodity  areas,  with  the  lead  Services  designated  in  1994,  are  listed 
below.  Descriptions  of  all  but  the  last  commodity  area  arc  extracted  from  Chapter  2  of 
the  1994  CBDP  Annual  Report  to  Congress.  The  description  of  modeling  and  simulation 
is  a  CBDP-focused  version  of  the  M&S  definition  in  DoD5000.59-M. 

•  Contamination  Avoidance  Army.  NBC  reconnaissance,  detection  and 
warning. 

•  Individual  Protection  -  Marines.  Protective  masks  and  protective 
clothing. 

•  Collective  Protection  -  Navy.  Shelters  for  command  posts,  rest  and 
relief,  vehicular  collective  protection,  and  safe  zones  aboard  ship. 

•  Decontamination  -  Air  Force.  Systems  used  to  reduce  or  eliminate 
hazards  to  units  after  exposure  to  NBC  contamination. 

•  Medical  Defense  -  Army.  Medical  prophylaxis,  pretreatment,  and 
therapies  used  to  protect  personnel  from  the  toxic  or  lethal  effects  of  NBC 
threat  agents. 

•  Modeling  and  Simulation  -  Navy.  The  use  of  models,  including 
emulators,  prototypes,  simulators,  and  stimulators,  to  develop  data  for 
making  decisions  related  to  CB  defense  activities. 
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Figure  3.  Inilial  CBDP  Management  Structure.  (From:  [5]) 
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Figure  4.  CBDP  Planning  and  Executing  Organizations  (From:  [6,  p.  23]) 
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The  Secretary  of  the  Army  was  designated  as  the  CBDP  Executive  Agent,  and  as 
such  is  “responsible  to  coordinate,  integrate,  and  review  all  Services’  CB  defense 
requirements  and  programs."  [7]  In  this  organizational  structure  the  individual  Services 
executed  programs  to  meet  Service- specific  and  DoD  requirements.  In  1997  the  Defense 
Threat  Reduction  Agency  (DTRA)  was  established,  and  given  responsibility  for  CBDP 
budgeting. 

These  changes  to  CB  defense  execution  within  DoD  were  an  improvement  over 
previous  practices  but  still  did  not  vastly  improve  efficiencies  of  operation  within  the 
CBDP.  In  Figure  4,  the  1999  GAO  report  GAO/NSIAD  99-159  recommended  that  an 
outcome-oriented  performance  plan  for  the  CBDP  be  developed  based  on  the  principles 
in  the  Government  Performance  and  Results  Act  (GPRA).  The  GAO  stated  that  the 
"Goals  of  the  CB  Defense  Program  Arc  Vague  and  Unmeasurable  and  Do  Not  Articulate 
Specific  Desired  Impacts,"  and  that  "CB  Defense  Program  Performance  Measures 
Emphasize  Activities  Rather  Than  Outcomes  and  Impacts."  Implementation  of  rcsults- 
based  management  was  inconsistent,  with  "The  Soldier  and  Biological  Chemical 
Command  (SBCCOM)"  as  "the  Only  RDT&E  Organization  to  Systematically  Apply 
Results  Act  Principles." 

These  lingering  deficiencies  prompted  major  organizational  changes  to  the  CBDP 
in  2001  with  the  creation  of  the  Program  Executive  Office  (PEO)  for  Chemical  and 
Biological  Defense,  which  became  the  Joint  PEO  for  Chemical  and  Biological  Defense 
(JPEO-CBD)  in  2003.  The  JPO  Biological  Defense  functions  were  incorporated  into  the 
PEOs.  This  new,  and  also  current,  organizational  structure  for  the  CBDP  is  shown  in 
Figure  5.  This  layout  is  designed  to  resolve  issues  inherent  in  previous  CBDP 
implementations,  such  as  single  service  control  of  programs,  rampant  duplicative  efforts, 
disconnects  between  the  development  and  test  organizations,  and  a  lack  of  oversight 
effectiveness. 
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Figure  5.  Current  CBDP  Management  Structure  (From:  [8,  p.  2]) 

This  change  maintains  the  commodity-based  structure  implemented  in  the  mid-1990s, 
with  two  modifications;  it  breaks  large  biological  detection  programs  into  a  separate  JPM 
and  adds  a  JPM  focused  on  installation  protection  within  the  United  States.  Since  CBDP 
funds  arc  centrally  budgeted  under  the  defense-wide,  rather  than  service-specific, 
category,  the  JPEO  and  JPMs  manage  programs  that  arc  truly  joint,  rather  than  single- 
scrvicc-driven  with  joint  ‘potential'  or  joint  ‘interest.' 

The  JPEO-CBD  includes  eight  Joint  Project  Managers  (JPMs),  primarily  aligned 
by  commodity  areas.  The  JPMs  with  their  functional  areas  of  responsibility  are: 

•  JPM  NBC  Contamination  Avoidance  (Army)-  NBC  detection, 
obscuration,  and  reconnaissance 

•  JPM  Biological  Defense  (Air  Force)  -  Biological  detection 

•  JPM  Collective  Protection  (Navy)  -  CBR  collective  protection 

•  JPM  Individual  Protection  (Marines)  -  CBR  individual  protection 

•  JPM  Decontamination  (Marines)  -  Individual  and  equipment 
decontamination 
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•  JPM  CB  Medical  Systems  (Army)  -  Medical  treatment,  vaccines,  and 
identification 

•  JPM  Information  Systems  (Army)  CB  C4I  and  models  and  simulations 

•  JPM  Guardian  (Army)  -  Installation  protection 

This  CBDP  organizational  layout  centralizes  management  of  all  Service  CB 
defense  acquisition  programs  under  the  JPEO-CBD.  This  structure  aligns  with  the 
existing  designation  of  the  Army  as  the  Executive  Agent  for  the  CBDP.  and  more 
importantly,  provides  a  single  reporting  chain  for  CBD  acquisition  programs  within  DoD. 
Requirements  are  handled  jointly  by  the  Joint  Requirements  Office  (JRO)  for  Chemical, 
Biological.  Radiological,  Nuclear  (CBRN)  defense  while  CB  Science  and  Technology  is 
separately  managed  by  the  Joint  Science  and  Technology  Office  (CBD)  within  the 
Defense  Threat  Reduction  Agency  (DTRA).  Chapter  1  of  the  2007  CBDP  Annual  Report 
provides  a  more  detailed  description  of  the  roles  and  responsibilities  of  all  CBDP 
organizations. 

Oversight  of  the  CBDP  as  an  entity,  with  the  CBDP  designated  as  an  ACAT  l-D 
program  (non-MDAP)  is  provided  by  the  Special  Assistant  for  CBD  and  Chemical 
Demilitarization  Programs  within  the  Assistant  to  the  Secretary  of  Defense  (Nuclear, 
Chemical,  and  Biological  Defense  Programs)  office.  [9]  Additionally,  individual 
programs  may  also  be  under  DOT&E  oversight  and/or  be  designated  as  Defense 
Acquisition  Executive  (DAE)  'Sentinel'  oversight  programs  within  the  CBDP. 

For  T&E,  the  Deputy  Under  Secretary  of  the  Army  (Operations  Research) 
(DUSA(OR))  has  responsibility  as  both  the  Army  and  the  CBDP  T&E  executive.  This 
enables  the  Secretary  of  the  Army  to  hold  both  the  materiel  development  executive,  the 
JPEO-CBD,  and  the  test  executive,  the  CBDP  T&E  Executive,  accountable  for  programs 
within  the  CBDP. 

Another  CBDP-uniquc  organization  is  the  Product  Director  of  Test  Equipment. 
Strategy,  and  Support  (PD-TESS)  within  the  office  of  the  JPM  NBC  Contamination 
Avoidance.  The  PD  TESS  is  chartered  "to  support  the  CBDP  T&E  Executive  in  matters 
of  test  infrastructure  development."  (8,  p.  63J  The  PD  TESS  is  tasked  to  manage 
upgrades  to  the  CBDP  test  infrastructure  (c.g.,  CB  agent  test  facilities)  through  FYI I  to 

address  existing  test  facility  limitations  to  effective  CB  test  execution. 
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In  summary,  the  one  and  a  half  billion  dollar  per  annum  CBDP  is  a 
congrcssionally-mandatcd,  uniquely-managed,  joint  enterprise  within  the  DoD.  It  has 
taken  a  deeadc  for  DoD  to  implement  an  effective  management  and  oversight  structure 
for  the  CBDP.  to  include  a  separate  CBDP  T&E  Executive,  an  organization  dedicated  to 
test  infrastructure  improvement,  and  to  provide  acquisition  program  responsibility  to  a 
single  truly  joint  organization. 

C.  INTRODUCTION  TO  DOD  TEST  AND  EVALUATION 

1.  The  Basis  for  Test  and  Evaluation  and  the  TEMP 

The  two  basic  reasons  that  T&E  is  performed  arc:  to  meet  legal  and  procedural 
requirements,  and  to  determine  whether  a  system  can  meet  the  performance  requirements 
levied  upon  it.  While  acquisition  programs  develop  many  test-related  documents,  it  is  the 
TEMP  that  is  the  keystone  for  program  T&E  efforts. 

a.  Legal  anti  Procedural  Basis  for  T&E 

The  formal  requirement  to  perform  T&E  within  DoD  is  derived  from 
multiple  sources,  including  public  laws,  statutes,  executive  orders,  and  public  policies. 
DoD  uses  these  sources  to  develop  the  policy  memorandums,  DoD  Directives  (DODDs), 
Instructions  (DODIs),  and  Regulations  (DOD-Rs)  that  implement  these  requirements. 
Per  DoD  Directive  5000.1,  it  and  DoD  Instruction  5000.2,  define  "management 
principles  and  mandatory  policies  and  procedures  for  managing  all  acquisition 
programs.”  These  details  include  definition  of  key  T&E  activities  and  documents,  and 
form  the  basis  for  the  development  of  lower-level  documents,  such  as  service-specific 
policies  and  implementation  guidance.  Examples  of  Sendee-specific  T&E 
implementation  documents  include  Army  Regulation  73-1,  DA  Pamphlet  73-1,  Air  Force 
Policy  Directive  99-1,  Secretary  of  the  Navy  Instruction  5000.2,  and  Marine  Corps  order 
3960.2B. 
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h.  Operational  Basis  for  T&E 


Even  if  the  legal  and  proecdural  requirements  did  not  exist,  there  still 
would  be  a  driving  force  behind  the  need  for  T&E  -  verifying  that  a  system  meets  the 
user’s  operational  requirements.  A  system  must  provide  a  technical  capability  that  the 
end  user  can  utilize  to  meet  a  mission  need,  i.e.,  the  system  must  be  operationally 
effective  and  suitable  for  the  warfighter.  The  user-driven  need  for  T&E  is  clearly 
summarized  by 

T&E  must  demonstrate  capabilities  today  that  will  be  needed  tomorrow 
in  combat.  When  a  system  is  called  for  combat  duty,  the  need  is 
immediate  and  there's  no  time  to  reconsider  if  the  system  will  operate  as 
designed.  Warfighters  need  assurance  that  when  they  risk  their  lives,  the 
systems  provided  will  do  the  job  for  which  they  were  designed.  [10.  p.  7] 

c.  The  Role  of  the  TEMP  in  T&E 

DoD  Instruction  5000.2  defines  the  requirement  for  a  TEMP,  and  per 
Table  E3.T2.,  requires  completion  of  the  initial  version  of  a  TEMP  for  an  acquisition 
program  by  MS  B,  with  updates  for  the  MS  C  and  the  FRP  decisions.  DA  Pamphlet  73- 1 
summarizes  the  role  of  the  TEMP  as  "the  basic  planning  document  for  all  life  cycle  T&E 
related  to  a  particular  system  acquisition  and  is  used  by  decision-making  bodies  in 
planning,  reviewing,  and  approving  T&E  activities.”  [  1 1.  p.  3]  The  DA  Pamphlet  further 
defines  the  TEMP  as  the  “overarching  T&E  document  used  by  the  T&E  community  to 
generate  detailed  T&E  plans  and  to  ascertain  schedule  and  resource  requirements 
associated  with  the  T&E  program."  [1 1,  p.  3] 

The  Program  Manager  (PM)  is  tasked  with  development  of  the  TEMP,  and 
specifically  with  the  technical,  or  developmental,  T&E  objectives  included  in  the  TEMP. 
The  Operational  Test  Agency  (OTA)  is  responsible  for  developing  the  OT&E  objectives 
within  the  TEMP.  Each  system  objective  is  described  in  the  TEMP,  typically  with 
technical  parameters  called  Critical  Technical  Parameters  (CTPs)  and  operational 
parameters  called  Critical  Operational  Issues  (COls),  Measures  of  Effectiveness  (MOE), 
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Measures  of  Suitability  (MOS),  or  Measures  of  Performance  (MOP).  Objectives 
categories  may  also  include  items  such  as  system  survivability  and  interoperability. 

Per  DA  Pamphlet  73-1,  CTPs  “are  measurable  critical  system 
characteristics  that,  when  achieved,  allow  the  attainment  of  operational  performance 
requirements.”  [1 1,  p.  4)  The  key  word  in  this  phrase  is  'allow,'  which  shows  that  CTPs 
define  the  technical  capability  that  is  required  to  make  the  desired  operational  result 
possible. 

In  contrast,  operational  test  parameters  arc  directly  based  upon  the  COls, 
COI  and  Criteria  (COlCs).  and  Key  Performance  Parameters  (KPPs)  from  the  system's 
requirements  document,  from  which  the  operational  test  evaluation  requirements,  the 
MOE,  MOS,  and  MOP,  arc  derived. 

Two  types  of  performance  measures,  technical  and  operational,  are  the  key 
parameters  used  during  T&E  efforts  to  determine  whether  a  system  can  meet  the  user’s 
operational  requirements.  Although  each  TEMP  is  program-specific,  all  "shall  describe 
planned  developmental,  operational,  and  live  fire  testing,  including  measures  to  evaluate 
the  performance  of  the  system  during  these  test  periods;  and  integrated  test  schedule;  and 
the  resource  requirements  to  accomplish  the  planned  testing."  (12,  p.  E5. 1.4.1]  Since  the 
TEMP  must  include  all  of  the  items  necessary  to  execute  the  T&E  efforts,  it,  therefore,  is 
the  one  acquisition  document  that  provides  the  information  necessary  for  a  comparative 
analysis  of  programs  across  acquisition  categories,  functional  areas,  and  time  periods. 

2.  Types  of  DOD  and  Commercial  Testing 

In  this  age  of  rapidly-advancing  technologies,  military  weapon  systems,  air  and 
ground  vehicles,  and  support  equipment  are  becoming  increasingly  difficult  to  acquire, 
maintain,  and  deploy  into  a  combat  environment.  The  demand  for  SMART  technology, 
super  battle-hardened  equipment,  and  embedded  software,  tangled  with  millions  of  line 
of  code,  pushes  the  envelope  of  defense  system  complexity.  The  resulting  increased  risk 
to  cost,  schedule,  and  performance  baselines  will  have  a  momentous  effect  upon  the  way 
DoD  conducts  T&E  to  verify  technical  parameters  and  validate  operational  performance. 
Currently,  DoD  conducts  single-tiered,  event-driven  Developmental  Testing  (DT)  and 
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Operational  Testing  (OT)  overlapping  with  the  contractor  testing  activities.  Embedded 
within  DT  and  OT  is  a  host  of  pre  and  post-Full  Rate  Production  (FRP)  sub-testing 
events.  Additionally,  many  program  offices  are  choosing  combined  DT/OT,  streamlining 
the  T&E  activities  in  efforts  to  mitigate  cost  and  schedule  risk.  Regardless  of  the 
strategics  employed,  program  offices  must  adhere  to  applicable  laws,  regulations,  and 
oversight  authorities  when  planning  and  conducting  the  numerous  T&E  efforts  in  support 
of  an  acquisition  program.  Consequently,  this  rigorous  and  expensive  process  enables  the 
fielding  of  reliable  products  and  promotes  user  confidence. 

Modem  developmental  testing  assesses  the  accomplishment  of  meeting  critical 
technical  parameters  and  determines  system  readiness  to  proceed  to  the  Initial 
Operational  Testing  (IOT).  [13,  p.  16)  Shown  in  Figure  1,  DT  is  typically  conducted 
throughout  the  Conceptual  Refinement,  Technology  Development,  and  System 
Development  and  Demonstration  phases,  even  continuing  up  through  FRP,  when 
involved  in  an  Evolutionary  Acquisition  (EA)  process.  DT  is  an  engineering  tool  that 
supports  critical  decisions  on  the  choices  of  design  plans,  materials,  manufacturing 
processes,  and  new  or  legacy  technology  integration.  Contractor  testing  is  typically 
conducted  during  the  Conceptual  Refinement  and  Technology  Development  phases,  and 
continually  monitored  by  the  program  office  in  preparation  for  testing  and  entrance 
criteria  for  major  milestone  decisions.  Essentially,  developmental  testing  minimizes 
design  and  integration  surprises,  which  in  some  eases,  can  be  fatal  to  an  acquisition 
program.  These  pre-FRP  tests  are  termed  Product  Verification  Test  (PVT),  Production 
Qualification  Test  (PQT),  Limited  User  Test  (LUT),  Live  Fire  Test  (LFT),  and  the  special 
testing  requirements  for  the  JPEO-CBD  programs.  Live  Agent  Test  (LAT).  For  a 
description  of  these  and  other  types  of  pre-FRP  testing  listed  below,  refer  to  Army 
Regulation  73-1. 

•  Research  effort  or  test 

•  Technical  feasibility  test 

•  Engineering  development  test 

•  Production  provc-out  test  (PPT) 

•  Software  qualification  test 
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•  Live  fire  test  for  vulnerability  &  lethality 

•  Logistics  demonstration 

•  C41/IT  interoperability  certification  test 

•  Software  development  test  (SDT) 

Operational  testing  provides  qualitative  and  quantitative  data  for  system 
assessments  and  evaluations,  and  determines  operational  effectiveness,  suitability,  and 
survivability  of  the  system  under  realistic  battlefield  conditions.  OT  is  controlled  by  an 
independent  agency,  using  production-representative  systems,  while  allowing  restricted 
contractor  involvement  to  minimize  bias,  and  essentially  supports  a  decision  to  pass  or 
fail  a  defense  system  before  it  goes  into  full-scale  procurement. 

Refer  to  AR  Regulation  73- 1  for  a  description  of  the  various  operational  tests. 

•  First  article  test 

•  Comparison  test 

•  Live  Fire  Test,  if  required  for  product  improvements  of  covered  systems 

•  Quality  conformance  (acceptance)  inspection 

•  Tests  in  support  of  post-deployment  software  support  (PDSS) 

•  C41/IT  interoperability  recertification  test 

•  Surveillance  test. 

•  Reconditioning  test. 

A  unique  set  of  requirements  for  Chemical.  Biological,  Radiological,  and  Nuclear 
defense  systems  is  Live  Agent  Testing.  LAT  refers  to  the  introduction  of  wcapons-gradc 
chemical  or  biological  contaminants,  also  known  as  Weapons  of  Mass  Destruction 
(WMD)  in  solid.  liquid,  and  vaporous  forms  to  the  system  being  tested.  Detection 
systems'  performance  is  quantitatively  characterized  by  measurements  associated  with 
the  speed  and  rate  of  detection  and  agent  concentration.  Additionally,  the  introduction  of 
common  battlefield  materials,  such  as  fuels,  solvents,  lubricants,  smokes  and  obscurants, 
etc.,  determines  the  detector's  potential  to  false  alarm. 

Testing  collective  protection  systems,  such  as  filtration  equipment,  in  mobile 
laboratories,  shelters,  and  combat  vehicles  determines  the  system’s  ability  to  safely 
collect  chemical  and  biological  agents  while  providing  breathable  air  to  the  occupants. 

Testing  individual  protection  systems,  such  as  Personal  Protective  Equipment  (PPE),  i.c. 
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masks,  gloves,  boots,  and  protective  clothing  determines  if  the  product  adequately  shields 
the  wearer  from  the  agent  for  a  given  mission  time  requirement.  Operational  testing  with 
live  agents  is  neither  practical,  safe,  nor  legal  in  the  continental  United  States.  Therefore, 
testers  and  evaluators  use  simulants  for  field  testing  with  actual  users.  Simulants  arc  an 
effective  testing  method  for  safely  replicating  CBRN  environments  in  military 
operations. 

3.  Program  Oversight 

One  of  the  factors  affecting  many  programs  is  oversight.  Oversight  is  "Senior 
Executive-level  monitoring  and  review  of  programs  to  ensure  compliance  with  policy  and 
attainment  of  broad  program  goals."  [10.  p.  66]  Programs  can  be  designated  for 
oversight  based  on  statutes  and  regulations,  and  by  designation  by  organizations  such  as, 
the  Under  Secretary  of  Defense  (Acquisition,  Technology,  and  Logistics)  (USD(AT&L)), 
DOT&E.  or  a  Service.  Oversight  can  be  specific  to  one  aspect  of  a  program,  such  as 
T&E,  or  general  in  nature,  where  all  key  program  documents  and  decisions  require 
oversight-organization  approval.  The  two  types  of  oversight  evaluated  during  this  study 
were,  the  Office  of  the  Secretary  of  Defense  (OSD)  T&E  oversight' DOT&E  oversight 
and  CBDP-specific  oversight. 

OSD  oversight  is  primarily  focused  on  T&E.  and  therefore  is  tightly  coupled  with 
the  congrcssionally-mandatcd  DOT&E  oversight  function.  DOT&E  ultimately  reports  to 
Congress,  but  is  organizationally  situated  within  DoD.  Within  USD(AT&L),  the 
Director  of  DT&E/Office  of  Defense  Systems  (DS)  designates  programs  for  OSD  T&E 
oversight  jointly  with  DOT&E,  in  consultation  with  Service  (and  CBDP)  T&E 
executives,  per  DOD1  5000.2.  The  2007  OSD  T&E  oversight  list,  although  marked  as 
For  Official  Use  Only  (FOUO),  is  publicly  accessible  at  the  following  website,  http, 
//www. acq.osd.mil/ssc/as/docs/2007_TE_Oversight_List. pdf.  The  OSD  oversight  list 
includes  ACAT  1  and  1AM,  II,  HI,  special  interest,  prc-MAIS,  and  pre-MDAP  programs, 
and  therefore  OSD.'DOT&E  oversight  is  not  unique  to  large  programs. 

Another  type  of  oversight,  called  "Sentinel,”  existed  within  the  CBDP  from  April 
2003  through  September  2007.  Sentinel  was  a  program-wide  oversight  activity 
performed  by  the  USD(AT&L)  on  CBDP  advanced  development  programs  that  had  the 
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cost,  complexity,  and  criticality  sufficient  to  warrant  monitoring  the  program’s  cost, 
schedule,  and  performance  "as  an  indicator  of  the  general  programmatic  health  of  the 
functional  area.”  [9J  Sentinel  oversight  and  OSD  oversight  were  not  mutually  exclusive. 
Therefore,  some  programs  within  the  CBDP  had  both  types  of  oversight  simultaneously. 
Sentinel  oversight  is  not  separately  evaluated  in  this  study,  as  all  Sentinel  systems 
reviewed  were  already  on  the  OSD  T&E  oversight  list. 

4.  Testing  and  Evaluation  Organizations 

Today,  many  DoD  acquisition  programs  utilize  numerous  testing  options 
available  from  the  different  services  and  private  industry.  While  a  substantial  portion  of 
developmental  testing  is  the  responsibility  of  the  contractor,  the  majority  of 
developmental  and  operational  testing  and  evaluation  of  DoD  systems  occurs  at  military 
T&E  facilities.  The  Department  of  Defense  acquisition  program  offices  arc  responsible 
for  early  coordination  with  these  organizations,  and  typically  dedicate  a  portion  of  the 
office  staff  to  the  responsibility  of  managing  the  program  test  and  evaluation  activities. 
The  areas  of  responsibility  characteristically  includes  defining  test  scope  and  objectives, 
documentation,  and  T&E  funding,  in  addition  to  coordinating  all  of  the  test  resources 
throughout  each  phase  of  the  acquisition  process.  [2,  p.  4-1]  Consequently,  the  PMO’s 
dedicated  T&E  activity  must  possess  a  comprehensive  knowledge  of  the  system  and  T&E 
process,  and  discuss  alternatives  associated  with  selecting  the  appropriate  support  activity 
for  an  effective  test  and  evaluation  strategy. 

This  section  familiarizes  the  reader  with  various  test  and  evaluation  activities 
throughout  DoD.  however,  limited  to  those  agencies  covered  under  the  Data  Summary 
and  Data  Analysis  chapters. 

a.  Army  T&E  Agencies 

The  Army  Test  and  Evaluation  Command  (ATEC)  is  headquartered  in 

Alexandria,  Virginia,  and  is  DoD's  single-most  integrated  testing  and  evaluation 

operation.  ATEC  comprises  integrated  evaluations  conducted  at  the  U.S.  Army 

Evaluation  Center  (AEC),  developmental  testing  at  the  U.S.  Army  Developmental  Test 

Command  (DTC),  and  operational  testing  at  the  U.S.  Army  Operational  Test  Command 
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(OTC).  On  October  1,  1999,  the  Army  finalized  the  earlier  decision  to  move 
developmental  and  operational  evaluation  into  a  single,  integrated  command.  Figure  6 
represents  the  Army’s  current  organizational  chart  that  identifies  the  major  commands 
and  subordinate  centers. 

The  Test  and  Evaluation  Command  (TECOM)  became  a  major 
subordinate  command  of  ATEC  and  was  redesignated  the  U.S.  Army 
Developmental  Test  Command  (DTC),  with  DTC  headquarters  remaining 
at  Aberdeen  Proving  Ground,  Maryland.  Also,  the  Test  and 
Experimentation  Command  (TEXCOM)  was  redesignated  the  U.S.  Army 
Operational  Test  Command  (OTC),  with  OTC  headquarters  remaining  at 
Fort  Hood.  Texas.  The  third  ATEC  subordinate  command  that  was 
redesignated  encompassed  both  the  Operational  Evaluation  Command  and 
the  Evaluation  Analysis  Center,  which  were  combined  to  form  the  new 
U.S.  Army  Evaluation  Center  (AEC).  (14] 

DTC  is  headquartered  at  the  Aberdeen  Proving  Ground  (APG),  Maryland, 
and  is  responsible  for  the  planning  and  management  activities  of  all  developmental 
testing  conducted  at  its  subordinate  centers.  DTC  also  offers  testing  services  to  other 
DoD  military  services  and  federal  agencies,  state  and  local  governments,  foreign  and 
allied  governments,  and  private  industry.  OTC  is  headquartered  at  Fort  Hood.  Texas,  and 
is  responsible  for  the  planning  and  management  of  all  operational  testing  in  the  areas  of 
equipment,  doctrine,  force  design,  and  training,  using  typical  soldiers  to  determine 
system  effectiveness,  suitability,  and  survivability. 

The  Army  Evaluation  Center  (AEC)  is  headquartered  in  Alexandria, 
Virginia,  while  maintaining  locations  at  APG,  Fort  Monmouth,  New  Jersey,  and  Fort 
Bliss,  Texas.  AEC  is  the  Army’s  total  system  evaluator  who  provides  to  the  final 
decision-makers  the  documentation  on  system  effectiveness,  suitability,  and  overall 
performance  during  testing.  AEC’s  early  involvement  in  the  acquisition  process  and 
working  closely  with  the  PMs  and  test  communities  ensures  that  T&E  strategics  and 
objectives  are  consistent  throughout  the  acquisition  program. 

The  Army  Materiel  Systems  Analysis  Activity  (AMSAA)  is  part  of  the 
Research,  Development,  and  Engineering  Command  (RDECOM)  and  is  located  at  the 
Aberdeen  Proving  Ground.  AMSAA  provides  the  DoD  acquisition  and  T&E 
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communities  with  analysis  support  and  solutions  to  materiel  and  logistics  systems  issues 
to  support  the  decision-making  process  for  acquiring  and  fielding  new  technologies. 

The  Army  Research  Laboratory  (ARL)  is  organized  under  the  Research. 
Development,  and  Engineering  Command  (RDECOM),  and  is  headquartered  at  the 
Adelphi  Laboratory  Center  in  Adelphi,  Maryland.  ARL  provides  the  Army  and  other 
DoD  organizations  with  applied  rcscareh.  analysis,  and  testing  of  weapons,  materials, 
human  systems  integration,  computational  sciences,  and  vehicle  and  equipment 
survivability  and  lethality. 

The  Aberdeen  Test  Center  (ATC)  is  located  at  the  Aberdeen  Proving 
Ground.  25  miles  northeast  of  Baltimore,  Maryland,  bordering  the  Chesapeake  Bay.  The 
installation  consists  of  79.000  acres  of  water  and  land  mass,  and  has  been  operating  since 
January  1918.  Today,  ATC  is  the  lead  DoD  test  center  for  automotive  (manned  and 
unmanned  ground  vehicles),  large  caliber  and  small  arms,  direct  fire  and  munitions,  and 
live  fire  vulnerability  and  lethality  testing. 

The  Aviation  Technical  Test  Center  (ATTC)  is  located  at  Cairns  Army 
Airfield.  Fort  Rucker,  Alabama.  ATTC  is  the  U.S.  Army’s  llight  test  activity  for 
airworthiness-qualification  of  both  rotary  and  fixed-wing  aircraft,  aviation  systems,  and 
aviation  support  equipment. 
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Figure  6.  U.S.  Army  Test  and  Evaluation  Command  Organization  Chart  (From:  (14]) 


Dugway  Proving  Ground  (DPG)  is  home  to  the  West  Desert  Test  Center 
(WDTC),  and  is  located  85  miles  southwest  of  Salt  Lake  City,  Utah,  in  the  Great  Salt 
Lake  Desert  on  798,855  acres  of  land.  WDTC  serves  as  the  nation's  chemical  and 
biological  defense  proving  ground,  and  also  conducts  reliability  and  survivability  testing 
for  meteorological,  smoke,  obscurants,  illumination,  and  munitions  systems. 

The  Yuma  Proving  Ground  (YPG)  and  the  Yuma  Test  Center  (YTC)  is  an 
ATEC  test  activity  located  near  the  Arizona-California  border,  approximately  24  miles 
north  of  the  city  of  Yuma.  Arizona.  In  the  context  of  this  JAP,  the  reference  of  YPG  or 
YTC  is  considered  to  be  the  same  test  activity.  YTC  is  a  multi-purpose  test  complex  for 
the  Army’s  desert  environment  testing,  with  only  about  three  inches  of  rainfall  per  year, 
encompassing  more  than  1,300  square  miles  of  test  range  area  and  2000  square  miles  of 
restricted  airspace.  The  test  center  olTcrs  realistic  testing  in  air  delivery,  ground  combat 
systems,  aircraft  armament,  unmanned  systems,  and  automotive  commodity  areas. 

The  Redstone  Technical  Test  Center  (RTTC)  is  one  of  the  U.S.  Army  Test 
and  Evaluation  Command  (ATEC)  activities  located  on  Redstone  Arsenal  near 
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Huntsville,  Alabama,  and  encompasses  over  14,000  acres  of  land.  RTTC  is  one  of  the 
eight  test  centers  that  comprise  the  Developmental  Test  Command.  RTTC’s  primary 
sendee  is  to  provide  advanced  testing  of  weapon  systems  for  the  Department  of  Defense 
and  its  contractors.  The  Center  provides  advanced  testing  in  two  major  areas;  flight- 
testing  of  small  rockets  and  guided  missiles  and  advanced  life-cycle  testing  for  weapon 
subsystems  and  components. 

The  Tropic  Regions  Test  Center  (TRTC)  is  located  in  Hawaii  and  Panama, 
while  other  test  facilities  are  currently  being  established  over  a  wide  geographic  area  for 
growing  tropical  testing  requirements.  TRTC  offers  real-world  tropical  effects  that 
include  insects,  bacteria,  high  temperature  and  humidity  conditions,  RAM  testing  of 
soldier  systems,  and  provides  mobility  testing  to  evaluate  system  ruggedness  and  small 
team  effectiveness. 

The  Cold  Regions  Test  Center  (CRTC)  is  located  in  the  mountainous 
regions  of  Alaska  encompassing  over  670,000  acres  of  land  with  restricted  airspace  to 
unlimited  altitude.  CRTC  accommodates  a  full  range  of  cold  weather  or  temperate 
climate  tests,  including  automotive  cold  start  capabilities,  mines,  explosives,  small  arms 
tests,  direct  fire  tests,  sensor  testing,  air  defense,  missile,  artillery',  smoke  and  obscurant 
tests,  and  mobility  testing.  CRTC  can  accommodate  indirect  fire  testing  with  the 
capability  of  observed  fire  to  30  km  and  unobserved  fire  to  50  km.  Indirect  fire  up  to  100 
km  is  accomplished  by  utilizing  ranges  near  Fort  Wainwright,  with  the  impact  on  Fort 
Grccly  areas. 

The  Army's  Electronic  Proving  Ground  (EPG)  is  located  at  Fort 
Huachuca,  Arizona,  and  encompasses  76,000  acres  of  land.  EPG’s  remote  location  is  an 
environment  free  of  radio- frequency  interference,  making  it  the  Army's  nucleus  for 
testing  Command,  Control,  Communications,  Computer,  and  Intelligence  (C4I) 
equipment  and  electronic  systems.  EPG  offers  developmental  testing  of  C41  systems. 
Unmanned  Aerial  Vehicles  (UAVs).  and  navigation  and  avionics  systems. 

The  White  Sands  Missile  Range  (WSMR»  is  located  20  miles  cast  of  Las 
Cruces,  New  Mexico,  and  is  where  the  United  States  successfully  tested  the  world’s  first 
atomic  bomb.  WSMR  covers  3,200  square  miles,  which  makes  it  the  largest  military 


installation  in  the  country.  WSMR  offers  a  variety  of  services  including  research  and 
development,  military  systems  and  commercial  product  assessments,  environmental 
testing,  computer  modeling,  and  opcn-air/ovcr-land  missile  testing. 

Fort  Indiantown  Gap  National  Guard  Training  Center  (F1TG)  is  located  in 
Annvillc,  Pennsylvania,  23  miles  cast  of  Harrisburg,  Pennsylvania.  F1TG  is  a  17,000 
acre  National  Guard  Training  site,  and  offers  military  and  law  enforcement  training, 
equipment  testing,  and  accommodates  a  variety  of  live-fire  ranges,  including  small  anus, 
shoulder-launched  rockets,  and  anti-armor  devices. 

h.  Navy  and  Marines  T&E  Agencies 

The  Operational  Test  and  Evaluation  Force  (COMPTEVFOR)  is  located  at 
the  Norfolk  Naval  Base  in  Norfolk,  Virginia.  COMPTEVFOR  is  the  Navy’s  sole 
independent  test  and  evaluation  activity,  reporting  directly  to  the  Chief  of  Naval 
Operations.  Slightly  dissimilar  to  the  Army’s  T&E  organizational  structure, 
COMPTEVFOR  combines  test  and  evaluation  within  each  division,  and  structured  along 
flexible  lines,  which  gives  primary  consideration  to  the  type  of  warfare  and  expertise. 
These  areas  are  the  Undersea  Warfare  Division.  Aviation  Warfare  Division,  C41  and 
Space  Division,  Surface  Warfare  Division,  and  Expeditionary  Warfare  Division.  Unless 
otherwise  specified,  the  use  of  COMPTEVFOR  in  this  JAP  refers  to  the  Navy  evaluation 
support.  Reference  to  other  Navy  activities  constitutes  testing  support. 

The  Marine  Corps  Operational  Test  &  Evaluation  Activity  (MCOTEA)  is 
located  at  Quantico,  Virginia,  and  is  the  United  States  Marine  Corps  (USMC)  OT&E 
authority  for  all  Marine  systems  and  equipment. 

The  Naval  Research  Laboratory'  (NRL)  is  located  in  the  southwest 
quadrant  of  Washington,  DC.  NRL  conducts  an  extensive  multidisciplinary  program  of 
technological  development,  scientific  research,  and  collaborative  testing  associated  with 
maritime  applications  including  advanced  materials,  support  equipment,  atmospheric 
systems,  and  space  sciences. 

The  Naval  Air  Systems  Command  (NAVA1R)  is  headquartered  at  the 
Patuxent  River  Naval  Base  in  Patuxent  River,  Maryland.  NAVA1R  is  one  of  the  Navy’s 
five  systems  commands  and  designs,  acquires,  manages,  and  maintains  the  Navy  air 

29 


systems  enterprise.  The  Naval  Air  Warfare  Center  Aircraft  Division  (NAWCAD),  also 
located  in  Patuxent  River,  is  one  of  the  subordinate  organizations  of  NAVA1R. 
NAWCAD  executes  life-cycle  support  in  the  areas  of  engineering  development, 
acquisition,  test  and  evaluation,  logistics,  and  training.  In  addition,  the  Fleet  Readiness 
Center  East,  at  the  Marine  Corps  Air  Station  Cherry  Point.  North  Carolina,  is  a  member 
of  the  NAVAIR  community,  primarily  providing  the  Marine  Corps  and  Navy  with 
aviation  maintenance  and  RDT&E  support.  Furthermore,  Cherry  Point  supports  DoD 
with  logistics  management  and  direct  field  support  for  a  variety  of  assignments. 

The  Naval  Sea  Systems  Command  (NAVSEA)  is  headquartered  in 
Washington,  DC.  and  is  the  largest  of  the  Navy's  systems  commands.  The  NAVSEA  and 
NAVAIR  organizations  function  is  a  similar  way,  where  NAVSEA  is  responsible  for  the 
Navy’s  entire  fleet  and  their  combat  systems.  The  Naval  Surface  Warfare  Center 
(NSW'C),  Dahlgrcn  Division,  is  located  in  Dahlgrcn,  Virginia,  and  is  structured  under 
NAVSEA.  Dahlgrcn  primarily  focuses  on  research,  development,  and  testing  on  warfare 
systems,  weapons,  munitions,  sensors,  software,  and  force  protection.  Dahlgrcn  also 
offers  drop  testing,  environmental  effects,  and  electromagnetic  energy  interference 
testing  and  evaluation.  The  Carderock  Division  is  located  in  West  Bcthcsda,  Maryland, 
focusing  on  survivability,  structures,  and  materials.  Carderock  also  specializes  in 
equipment  shock  hardening,  vibration  testing,  hull  design,  and  shipboard  survivability. 
The  Navy  Clothing  and  Textile  Research  Facility  (NCTRF)  is  a  NAVSEA  asset  for  the 
RDT&E  of  personal  protective  equipment,  damage  control,  and  fire  protection 
engineering.  NCTRF  hosts  a  variety  of  testing  capabilities,  including  a  dynamic  fire  test 
chamber,  a  pressurized  steam  test  chamber,  and  a  thermal  oven  chamber  specifically 
designed  for  the  testing  of  PPE.  The  Navy's  Space  and  Naval  Warfare  Systems 
Command  (SPAWAR)  is  located  in  San  Diego,  California,  and  provides  DoD  with 
RDT&E  and  logistical  support,  acquisition  management  of  joint  systems  associated  with 
C4I  and  space  technologies. 

The  Naval  Operational  Medicine  Institute  (NOMI)  is  located  in  Pensacola, 
Florida.  NOMI  participates  in  doctrine  development,  experimentation,  and  testing  in 
support  of  naval  operations  and  naval  survival  systems. 
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The  Naval  Safety  Center  (NSC)  is  located  in  Norfolk,  Virginia,  and 
provides  the  Secretary  of  the  Navy  (SECNAV)  with  safety-related  management  for  all 
Naval  and  Marine  Corp  issues.  NSC  also  provides  the  DoD  acquisition  community  with 
system  evaluations,  risk  identification,  and  hazard  analysis  documentation  for  programs 
under  development. 

Naval  Station  Mayport,  Florida,  is  adjacent  to  Jacksonville  and  situated  on 
3,400  acres  of  land.  Mayport  provides  naval  fleet  port  accommodations,  operational  and 
training  activities  for  anti-submarine  warfare,  and  also  provides  on-board  operational 
testing  and  training  support  through  the  Afloat  Training  Group  (ATG). 

The  Marine  Corps  Systems  Command  (MARCORSYSCOM), 
headquartered  at  Marine  Base  Quantico,  Virginia,  and  the  home  of  Marine  training  and 
doctrine  since  1917.  MARCORSYSCOM  is  the  principal  activity  for  acquisition,  testing, 
and  sustainment  of  systems  and  equipment  used  by  the  Marine  forces. 
MARCORSYSCOM  products  encompass  the  full  spectrum  of  warfighter  needs, 
including  armor,  combat  and  support,  C41,  ground  transportation,  and  sensor  systems. 

c.  Air  Force  T&E  Agencies 

The  Air  Force  Operational  Testing  and  Evaluation  Command  (AFOTEC) 
is  headquartered  at  Kirtland  Air  Force  Base  (AFB),  New  Mexico,  which  is  also  the  site  to 
one  of  five  Air  Force  T&E  detachments;  Eglin  AFB-Florida.  Peterson  AFB-Colorado, 
Edwards  AFB-Califomia,  and  Nellis  AFB-Nevada.  AFOTEC  provides  multi-service  test 
and  evaluation  support  in  the  areas  of  weapons,  support  equipment,  C4ISR,  space,  missile 
defense,  and  aircraft. 

The  Air  Force  Medical  Operations  Agency  (AFMOA)  is  located  at  Brooks 
Air  Force  Base,  San  Antonio,  Texas.  AFMOA  is  responsible  for  the  Air  Force’s 
aerospace  medicine,  biocnvironmental  engineering  and  biocnvironmental,  and  is  the 
consultant  lead  for  aerospace  medicine  and  preventive  medicine,  clinical  research, 
medical  resources,  and  chemical,  biological,  and  radiation  protection. 

Edwards  Air  Force  Base,  California,  is  home  to  the  Air  Force  Flight  Test 
Center  (AFFTC).  AFFTC  conducts  flight  and  ground  testing  of  aircraft  and  support 
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equipment,  weapons  systems,  software,  and  additionally  performs  Modeling  and 
Simulation  (M&S)  for  the  Air  Force. 

Wright- Patterson  Air  Force  Base  (WPAFB),  Ohio,  is  home  to  the  Air 
Force  Materiel  Command  (AFMC)  conducting  RDT&E,  acquisition  management,  and 
logistical  operations  for  Air  Force  systems.  WPAFB  is  also  home  to  the  Air  Force 
Research  Laboratory  (AFRL),  Aeronautical  Systems  Center,  and  National  Museum  of  the 
United  States  Air  Force. 

Prior  to  its  closure  in  2001,  McClellan  Air  Force  Base  in  Sacramento, 
California,  was  home  to  the  Sacramento  Air  Logistics  Center,  situated  on  3,778  acres  of 
land,  and  conducts  a  variety  of  missions  including  the  maintenance  and  management  of 
US  Air  Force  electrical  components,  communications,  electronics  systems,  fluid  drive 
accessories,  and  tactical  shelters. 

Hill  Air  Force  Base  is  located  in  northern  Utah,  operating  and  maintaining 
the  Utah  Test  and  Training  Range.  The  range  provides  open  air  testing  services  for  a 
variety  of  needs,  including  weapons  testing,  large  force  training  exercises,  and  range 
support  for  Air  Force  operational  test  and  training  programs. 

The  Air  Mobility  Command  at  Pope  Air  Force  Base,  North  Carolina,  is 
responsible  for  delivering  humanitarian  effects  and  expeditionary  aid,  aerial  refueling, 
and  aerial  combat  support.  Additionally,  Pope  AFB  groups  provide  support  aircraft  to 
training  exercises  and  operational  testing. 

d.  The  Commercial  Sector 

Wylc  Laboratories  Inc.  is  headquartered  in  El  Segundo,  California,  with 
more  than  30  facilities  nationwide.  Wylc  labs  provide  test  and  evaluation  support  to 
DoD.  NASA,  and  commercial  customers  with  vibration  systems  testing,  environmental, 
and  structural  testing.  Wylc  also  provides  support  in  acquisition  management, 
engineering,  and  space  sciences. 
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Alion  Science  &  Technology  R&B  Laboratory  operates  as  an  independent 
testing  lab  for  electro-magnetic  emission  and  interference,  and  radio  frequency 
quantification  supporting  the  automotive  industry,  various  commercial  aircraft  and 
avionics  products,  and  military  applications. 

Battclle  is  a  large  international  science  and  technology  business, 
headquartered  in  Columbus.  Ohio,  possessing  RDT&E  capabilities  in  energy 
conservation,  lab  management,  heath  sciences,  and  national  security  and  defense. 
Additionally,  Battclle  supports  defense  projects  associated  with  chemical  and  biological 
detection  and  protection  equipment  for  the  joint  services. 

Lincoln  Laboratory  at  the  Massachusetts  Institute  of  Technology  (MIT)  is 
a  federally  funded  R&D  center  for  applied  science  and  technology,  specializing  in 
advanced  electronics  applications  associated  with  air  and  missile  defense, 
communications,  tactical  and  space  surveillance,  and  air  traffic  control  systems. 
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III.  DATA  SUMMARY 


A.  INTRODUCTION 

The  researcher's  approach  to  data  collection  is  to  gather  information  for  the 
purposes  of  both  elimination  and  discovery.  This  approach  is  based  upon  a  philosophy 
that  there  are  no  erroneous  samples  of  data.  The  many  significant  discoveries  that  have 
been  made  by  exploiting  seemingly  useless  data  for  other  purposes,  supports  this  mindset. 
The  tester’s  approach  to  data  collection  is  more  narrowly  focused;  to  validate  a  known  or 
desired  parameter  through  measurement. 

Data  was  collected  for  this  research  project  using  the  philosophy  of  a  researcher, 
with  the  constraint  that  each  data  clement  must  be  applicable  to  multiple  programs.  This 
constraint  minimized  the  collection  and  presentation  of  data  that  had  no  possibility  of 
helping  to  answer  the  primary  and  secondary  research  questions.  The  data  presented  in 
this  chapter  was  collected  from  twenty-three  acquisition  programs.  These  programs 
cover  the  breadth  and  depth  of  acquisition  possibilities,  including,  ACAT  I,  II,  and  III; 
T&E  oversight  and  non-oversight;  joint  service  and  Army-only;  CBDP  and  non-CBDP; 
Milestone  B  through  Full  Rate  Production;  and  both  current  and  legacy  acquisition 
programs. 

The  majority  of  the  data  contained  in  this  chapter  is  directly  traceable  to  data 
elements  extracted  from  the  twenty-three  program  TEMPs  recorded  in  the  List  of 
References.  Other  documents,  such  as  the  program's  performance  specification  were 
used  to  fill  TEMP  data  gaps  when  possible.  The  one  derived  category  of  data  element 
presented  in  this  chapter  is  the  quantifiable  data  for  CTPs  and  MOE-MOS.  The 
derivation  method  for  the  quantifiable  data  types  is  defined  in  Chapter  IV. 

The  data  presented  in  this  chapter  is  categorized  as,  principal  data;  secondary 
data;  and  unavailable,  inconsistent,  and/or  deficient  data.  Principal  data  is  the  primary 
information  source  for  analysis  in  Chapter  IV.  Secondary  data  only  indirectly  supports 
answers  to  the  research  questions,  while  the  last  category  of  data  encompasses 
information  that  cither  was  not  available  or  could  not  be  effectively  analyzed. 
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B.  DATA  PARAMETERS 

1.  Principal  Data 

Principal  data  is  defined  as  tangible  and  consistently  available  information 
associated  with  each  program.  This  information  provides  administrative  and  technical 
details  for  PM  planning  and  estimating  purposes,  and  is  equally  valuable  for  strategic 
T&E  preparation. 

Programmatic  and  Technical  data  consists  of  Critical  Technical  Parameters, 
Measures  of  Effectiveness,  and  Measures  of  Suitability.  CTPs  describe  system  hardware 
and  software  characteristics  and  thresholds  contained  in  the  CDD,  and  must  be  consistent 
with  system  operational  performance  requirements.  Critical  technical  parameters  arc, 

measurable  critical  system  characteristics  that,  when  achieved,  allow  the 
attainment  of  desired  operational  performance  capabilities.  They  arc  not 
user  requirements.  Rather,  they  are  technical  measures  derived  from 
desired  user  capabilities.  Failure  to  achieve  a  critical  technical  parameter 
should  be  considered  a  reliable  indicator  that  the  system  is  behind  in  the 
planned  development  schedule  or  will  likely  not  achieve  an  operational 
requirement.  [15.  p.  9.  lOj 

Table  1  presents  the  total  number  of  CTPs  and  quantifiable  CTPs  for  each  program. 

MOEs  are  assessments  of  operational  performance  that  evaluate  the  degree  of 
successfulness  of  the  system  with  respect  to  the  objective  of  the  mission  or  operation.  [2, 
p.  B- 1 1 J  Specifically,  the  MOE  assesses  the  mission  accomplishment  of  a  system  when 
used  by  representative  personnel  in  the  environment  planned  or  expected  (c.g.,  natural, 
electronic,  threat  etc.)  for  operational  employment  of  the  system  considering 
organization,  doctrine,  tactics,  survivability,  vulnerability,  and  threat.  [15,  p.  9.10J  A 
MOS  is  an  operational  assessment  that  evaluates  a  system’s  capacity  of  being 
satisfactorily  fielded  in  an  environment  considering  issues  pertaining  to  reliability, 
availability,  maintainability,  logistical  support,  compatibility,  interoperability,  training, 
human  factors,  safety,  documentation,  transportability,  wartime  usage  rates,  manning 
requirements,  and  natural  and  environmental  effects  and  impacts.  [15,  p.  9. 10]  Table  2  of 
this  chapter  presents  MOE,  MOS,  and  quantifiable  MOE/MOS  data  sets.  The  Defense 
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Acquisition  Guidebook  suggests  that  program  managers  must  develop  a  crosswalk  for 
operational  effectiveness  and  suitability  parameters  and  constraints 

to  those  used  in  the  Analysis  of  Alternatives  (AoA),  and  include 
manpower,  personnel,  training,  software,  computer  resources, 
transportation  (lift),  compatibility,  interoperability  and  integration. 
Information  Assurance  (1A),  Electromagnetic  Environmental  EITccts,  and 
Spectrum  Supportability.  The  focus  is  on  operational  capabilities,  not 
design  specifications  such  as  weight  and  size.  (15,  p.  9. 10] 

Although  seemingly  trivial,  the  TEMP  page  count.  Table  3,  is  essentially  the  total 
number  of  pages,  excluding  the  cover  and  signature  pages,  and  the  number  of  pages  in 
sections  I  through  V.  Surprisingly,  this  information  gives  useful  insight  into  which 
programs  followed  the  DoD  recommended  TEMP  page  count  guidelines  and  formulates 
insight  into  the  level  of  documentation  effort. 

Many  DoD  acquisition  programs  utilize  numerous  organizations  for  T&E  support 
depending  upon  system  requirements,  organizational  past  performance,  and  range 
availability,  to  name  a  few.  The  programs  investigated  in  this  JAP  benefited  from  the 
diverse  expertise  of  multi-service  and  private  testing  organizations  briefly  introduced  in 
Chapter  II,  consisting  of  the  Army,  Navy,  and  Air  Force  testing  and  evaluation 
commands,  in  addition  to  a  host  of  industry  and  university  locations,  such  as  Lincoln 
Labs  at  MIT.  Tables  5  through  12  illustrate  the  various  test  and  evaluation  organizations 
utilized  by  each  program. 

2.  Secondary  Data 

Selected  programmatic  and  service-specific  information  concerning  each  program 
was  collected  for  reference  and  to  help  bridge  gaps  in  formulating  an  enhanced 
comprehensive  analysis.  This  section  consists  of  the  total  number  of  estimated  personnel 
required  throughout  DT  and  OT,  COls,  the  TEMP  date,  and  program  oversight 
requirements. 
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The  TEMP  dale,  shown  in  Table  4,  is  the  official  dale  of  document  authorization 
and/or  the  current  documented  revision.  Each  program’s  TEMP  date  was  collected  for 
comparative  purposes  in  an  attempt  to  discover  a  relationship  of  pre/post-ycar  2000 
TEMPs  to  other  principal  and  secondary  data. 

All  AC  AT  I,  and  selected  ACAT  II  programs,  require  OSD  and  DOT&E 
oversight  as  described  in  the  Background  Chapter.  However,  many  ACAT  111  programs 
arc  not  burdened  with  this  acute,  yet  necessary  supervision.  Three  of  the  seventeen 
ACAT  III  programs  investigated  required  some  level  of  oversight.  Interestingly,  these 
three  programs  arc  detection  systems. 

Critical  Operational  Issues  arc  the  key  operational  effectiveness  or  suitability 
issues  that  must  be  scrutinized  during  OT&E  to  determine  a  system’s  capability  of 
successfully  executing  the  intended  mission.  COIs  arc  typically  represented  as  questions. 
The  total  number  of  COIs  for  each  program  are  listed  in  Table  2.  The  number  of  COIs 
demonstrated  no  stand-alone  significance.  However,  the  development  of  a  mathematical 
analysis  of  effectiveness  and  suitability  issues  enabled  a  direct  comparison  of  programs 
across  ACAT  levels  and  is  investigated  in  Chapter  IV. 

3.  Unavailable,  Inconsistent,  and/or  Deficient  Data 

Certain  program  data  was  initially  perceived  as  indisputable  evidence  supporting 
the  questions  in  this  JAP.  The  most  difficult  and  disappointing  sets  of  data  was  program 
funding.  The  original  methodology  was  to  focus  all  other  supporting  data  with  the 
amount  of  funding  each  program  received  and  determine  if  the  smaller  programs 
typically  endured  the  same  T&E  burden  while  having  less  funding  and  resources.  It  was 
discovered  that  the  funding  requirements  estimated  in  the  TEMPs  only  reported 
combined  RDT&E  funding.  A  valid  funding  comparison  was  not  possible  due  to  the 
drastic  differences  in  system  complexity  and  varying  production  quantities.  High 
complexity  programs  with  less  technical  maturity  will  likely  have  more  research  funding. 
This  budgeting  information  must  be  a  separate  line  item  of  the  total  RDT&E  estimate  in 
order  to  accurately  determine  the  T&E  segment.  Additionally,  the  RDT&E  funding 
reported  in  the  Program  Objective  Memorandum  (POM)  drastically  differed  from  the 


data  gathered  from  TEMPs.  For  example,  POM  funding  data  includes  congressional 
plus-ups  and  reductions,  program  re-structuring,  and  out  of  control  cost  increases, 
rendering  a  sensible  comparison  virtually  unattainable.  Additionally,  the  TEMP  funding 
data  were  skewed  based  upon  the  various  acquisition  phases. 

Another  data  set  expelled  from  the  analysis  was  the  total  number  of  required 
personnel  estimated  throughout  testing  due  to  incompleteness  and  inconsistencies  across 
the  programs.  Some  TEMPs  were  exceedingly  specific,  listing  the  various  multi-service 
requirements,  logistical  support  personnel,  the  number  of  evaluators,  etc.  Some  TEMPs 
explicitly  called  for  precise  numbers  of  officers  and  enlisted  personnel  by  their  actual 
Military  Occupational  Specialty  (MOS),  the  number  of  battalions,  companies,  teams, 
shipboard  and  Operational  Force  (OPFOR)  participants  during  specific  events.  Other 
programs  were  exceedingly  broad  in  their  requirements,  merely  mentioning  the  need  for 
test  support  personnel  and  OT  participants. 

Likewise,  an  impartial  comparison  of  the  number  of  test  articles  and  supporting 
equipment  estimated  in  the  program  TEMP’s  emerged  from  the  data  set.  It  is  not 
practical  to  compare  a  program  requiring  nine  vehicles  to  a  chemical  detection  system’s 
requirement  of  250  detectors,  or  even  5,750  masks  for  a  mask  program.  Additionally, 
some  programs  were  very  specific  in  listing  supporting  equipment  necessary  during 
testing.  For  example,  the  Joint  Service  General  Purpose  Mask  (JSGPM)  program 
required  a  video  reduction  station,  one  portable  weather  station,  two  thousand  vision 
correction  systems,  various  video  and  audio  components,  and  twenty  mask  leakage 
testers.  In  contrast,  the  Joint  Chemical  Agent  Detector  (JCAD)  program  required 
numerous  vehicles,  including  sixteen  High  Mobility  Multipurpose  Wheeled  Vehicles 
(HMMWVs),  three  Bradley  Fighting  Vehicles  (BFV),  six  utility  vehicles,  and  six  ten- 
passenger  vans.  The  JCAD  TEMP  merely  mentioned  the  need  for  35  handheld  radios 
and  two  Single-Channel  Ground-Air  Radio  System  (S1NGARS)  radio  sets.  This  disparity 
in  requirements  was  consistently  evident  across  the  entire  set  of  TEMP  documents  and 
did  not  demonstrate  the  continuity  required  for  the  analysis  in  Chapter  IV. 

The  TEMP  milestone,  shown  in  Table  4,  is  the  major  milestone  at  which  the 
program  achieved  authorization,  or  the  current  published  document  version.  Knowing 
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ihc  acquisition  phase  at  TEMP  currency  is  fundamentally  significant  for  demonstrating 
the  breadth  of  data  sets  across  numerous  acquisition  phases,  dissimilar  program  types, 
and  different  ACAT  levels. 

One  effective  method  for  comparing  test  burden  is  by  associating  the  number  of 
test  events  conducted  or  proposed  in  the  program’s  Event  Design  Plan  (EDP>  to  another 
test  parameter.  However,  the  representative  data  on  Army-led  test  events  arc  available 
only  through  ATEC,  via  the  ATEC  Decision  Support  System  (ADSS)  website.  Test 
event  plans  and  results  collected  from  ADSS  do  not  thoroughly  represent  the  actual  total 
number  of  events  since  many  programs  utilized  other  services  and  private  laboratories  for 
T&E  support,  and  these  organizations  were  not  enthusiastic  in  providing  this  type  of 
program  information. 

Originated  in  the  CDD,  previously  called  the  Operational  Requirements 
Document  (ORD),  Key  Performance  Parameters  are  a  system’s  minimum  set  of  user- 
defined.  functional  and  operational  requirements  necessary  to  fulfill  the  conditions  and 
constraints  stated  in  the  Mission  Needs  Statement  (MNS),  and  pilots  the  efforts  of  the 
System  Development  and  Demonstration  Phase.  These  KPPs  must  be  approved  prior  to 
entrance  into  the  SDD  Phase,  and  "may  be  refined,  with  the  approval  of  the  requirements 
authority,  as  conditions  warrant."  [15,  p.  3. 7.2. 3]  KPPs  arc  a  critical  subset  of  the 
performance  parameters,  arc  included  in  the  performance  portion  of  the  APB,  and 
represent  the  most  significant  capabilities  or  characteristics  for  a  given  system.  Failure  to 
meet  the  minimum  or  threshold  value  of  performance  can  be  cause  for  program  to  be 
reevaluated  or  terminated.  The  total  number  of  KPPs  for  each  program  is  listed  in  Table  1 . 

C.  HAPTER  SUMMARY 

The  primary  focus  of  this  chapter  was  to  define,  present,  and  initially  investigate 
all  of  the  collected  data,  and  determine  whether  principal  and  secondary  data  is  beneficial 
to  the  development  of  a  mathematical  approach,  or  basis  for  elimination  in  the  data 
analysis  for  answering  the  questions  presented  in  this  JAP.  Furthermore,  selected 
programmatic  and  sendee-specific  information  concerning  each  program  were  collected 
for  reference  and  to  help  bridge  gaps  in  formulating  an  enhanced  comprehensive  analysis. 
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Moreover,  the  elementary  process  of  organizing  the  data  into  practical  categories  exposed 
inconsistencies  and  inadequacies,  such  as  funding,  test  expendables,  special  equipment, 
and  manpower  estimates  that  were  especially  difficult  comparisons  and  provided  little 
insight  to  the  objectives  herein. 


DATA  PARAMETER 

TOTAL 
NO.  KPP 

TOTAL 
NO.  CTP 

TOTAL  NO. 
QUANTIFIABLE  CTP 

ACAT 

PROGRAM 

FCS 

7 

13 

6 

I 

NBCRV 

2 

25 

16 

STRYKER 

5 

19 

11 

JBPDS 

5 

14 

6 

II 

JCAD 

0 

14 

11 

JSLNBCRS 

0 

22 

9 

AC  AD  A 

0 

17 

13 

ALSI-1 

4 

4 

3 

BIDS 

4 

18 

6 

CBPSS 

6 

24 

10 

FOX 

0 

9 

7 

JBAIDS 

3 

7 

5 

JBSDS 

4 

21 

14 

JPACE 

2 

20 

10 

III 

JSC  PM 

5 

41 

16 

JSLIST-GLOVE 

8 

27 

16 

JSLSCAD 

6 

29 

21 

JSTDS 

5 

61 

30 

LVOSS 

0 

1 1 

4 

M45 

0 

18 

15 

M56 

0 

12 

12 

M6 

0 

7 

3 

SHIPBRD-CPS 

0 

1 1 

9 

Table  1.  KPPs  and  Quantifiable  CTPs. 
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DATA  PARAMETER 


TOTAL  NO. 
MOE/MOS 

TOTAL  NO. 

QUANTIFIABLE  MOE/MOS 

ACAT 

PROGRAM 

FCS 

35 

I 

I 

NBCRV 

38 

5 

STRYKER 

24 

6 

JBPDS 

29 

8 

II 

JCAD 

69 

34 

JSLNBCRS 

19 

4 

ACADA 

39 

26 

ALSI-I 

5 

3 

BIDS 

12 

6 

CBPSS 

14 

7 

FOX 

15 

12 

JBAIDS 

15 

10 

JBSDS 

12 

6 

JPACE 

44 

14 

III 

JSGPM 

55 

16 

JSLIST-GLOVE 

25 

13 

JSLSCAD 

17 

11 

JSTDS 

51 

19 

LVOSS 

21 

8 

M45 

13 

8 

M56 

27 

10 

M6 

27 

10 

SHIPBRD-CPS 

6 

5 

Table  2.  Total  Number  of  MOEs  and  MOSs. 
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DATA  PARAMETER 

NO. 
CO  I 

NO.  TEST 
ARTICLES 

NO. 

TEMP 

PAGES 

NO.  TEMP 

PAGES  SEC. 

I-Y 

PROGRAM 

FCS 

6 

NA 

482 

93 

NBCRV 

13 

9 

78 

58 

STRYKER 

9 

NA 

134 

57 

JBPDS 

4 

102 

136 

67 

JCAD 

2 

250 

160 

95 

JSLNBCRS 

3 

25 

74 

62 

AC  A  D  A 

3 

140 

54 

25 

ALSI-1 

4 

9 

71 

30 

BIDS 

3 

12 

44 

37 

CBPSS 

5 

18 

67 

43 

FOX 

3 

23 

30 

29 

JBAIDS 

2 

20 

110 

70 

JBSDS 

3 

14 

108 

40 

JPACE 

4 

390 

102 

75 

JSC.  PM 

3 

5750 

84 

46 

JSLIST-GLOVE 

3 

678 

105 

51 

JSLSCAD 

4 

29 

76 

42 

JSTDS 

3 

250 

83 

61 

LVOSS 

2 

1772 

20 

16 

M45 

4 

800 

41 

35 

M56 

2 

18 

25 

17 

M6 

2 

127 

23 

18 

SHIPBRD-CPS 

2 

4 

27 

23 

Tabic  3.  COIs  and  TEMP  Page  Counts. 
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DATA  PARAMETER 

TOTAL 

MANPOWER 

TEMP 

DATE 

TEMP 

MILESTONE 

OSD/DOT&E 

OVERSIGHT 

ACAT  PROGRAM 


FCS 


NBCRV 


STRYKER 


JBPDS 


JCAD 


JSLNBCRS 


AC  ADA 


ALSI-1 


BIDS 


CBPSS 


FOX 


JBAIDS 


JBSDS 


JPACE 


JSGPM 


JSLIST-GLOVE 


JSLSCAD 


JSTDS 


LVOSS 


M45 


M56 


M6 


SHIPBRD-CPS 


23-May-2003 


9- Dec- 1997 


30-0ct-2006 


12-Scp-2001 


7-Jun-2002 


Apr- 1998 


28-Apr-2004 


1 1 -Mar-2005 


10-May-2004 


30-\lar-2005 


3-Fcb-2006 


28-Feb-2005 


Oct- 1998 


Feb- 1994 


Jul-2005 


Scpt-2000 


23-Mar- 1994 


MSIII 


LRIP 


FRP 


SI-IPRC 
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Tabic  5.  Evaluation  Organizations  Utilized. 
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ACAT 


PROGRAM 


ATC 


FCS 

ss: 

I 

NBCRV 

X 

STRYKER 

X 

JBPDS 

SB 

II 

JCAD 

mm 

JSLNBCRS 

X 

ACADA 

KM 

ALSI-I 

X 

BIDS 

X 

CBPSS 

X 

FOX 

ESI 

JBAIDS 

JBSDS 

X 

JPACE 

III 

JSGPM 

SB 

JSLIST- 

GLOVE 

19 

JSLSCAD 

X 

JSTDS 

X 

LVOSS 

ESI 

M45 

M56 

X 

M6 

ESI 

SHIPBRD-CPS 

Tabic  6. 


ARMY  TESTING  ORGANIZATION 


DTC 

DPG 

EPG 

YPG 

WSMR 

CRTC 

ARE 

X 

X 

ES 

X 

X 

X 

EB 

X 

X 

X 

X 

X 

SB 

X 

X 

X 

X 

X 

X 

X 

X 

X 

ES 

X 

X 

X 

X 

X 

X 

X 

EB 

X 

X 

X 

EB 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

ES 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

ES 

X 

X 

X 

X 

X 

ES 

X 

X 

X 

X 

mm 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

ons  Utilized. 
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Tabic  7.  Additional  Army  Test  Organizations. 


47 


Tabic  8.  Navy  Test  Organizations  Utilized. 
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ACAT 

PROGRAM 

> 

NOMI 

I 

FCS 

NBCRV 

STRYKER 

II 

JBPDS 

JCAD 

JSLNBCRS 

III 

ACADA 

ALSI-1 

BIDS 

CBPSS 

FOX 

JBAIDS 

JBSDS 

JPACE 

X 

JSGP.M 

JSLIST- 

GLOVE 

JSLSCAD 

JSTDS 

LVOSS 

M45 

M56 

M6 

SHIPBRD-CPS 

Tabic  9.  Additional  Navy  Test 


TESTING  ORGANIZATION 


NCTRF 

NSC 

NAWCAD 

SPA  WAR 

X 

X 

X 

X 

rations. 
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ACAT 

PROGRAM 

All 

Martin  El» 

1 

FCS 

NBCRV 

STRYKER 

II 

JBPDS 

JCAD 

JSLNBCRS 

III 

ACAD  A 

ALSI-I 

BIDS 

CBPSS 

FOX 

JB  AIDS 

JBSDS 

JPACE 

JSCPM 

JSLIST- 

GLOVE 

JSLSCAD 

JSTDS 

LVOSS 

M45 

M56 

M6 

X 

SHIPBRD-CPS 

Table  1 1.  Additional  Air  Force  Test 


FORCE  TESTING  ORGANIZATION 


ectronics 

WPAFB 

Cherrv  Point 

AFMSA 

X 

X 

X 

;anizations. 
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ACAT 

PROGRAM 

PRIVATE 

Alion 

Labs 

Batti 

I 

FCS 

NBCRV 

STRYKER 

II 

JBPDS 

X 

X 

JCAD 

JSLNBCRS 

III 

ACAD  A 

ALSI-1 

BIDS 

CBPSS 

FOX 

JBAIDS 

JBSDS 

.1  PACE 

X 

JSGPM 

JSLIST- 

GLOVE 

X 

JSLSCAD 

JSTDS 

X 

LVOSS 

M45 

M56 

M6 

X 

SHIPBRD-CPS 

Tabic  12.  Private  Test  Organizations  Utilized. 
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IV.  DATA  ANALYSIS 


A.  INTRODUCTION 

This  chapter  analyzes  the  data  presented  in  Chapter  III  to  develop  answers  to  the 
primary  and  secondary  research  questions.  The  primary  research  question  is,  “Is  the  Test 
and  Evaluation  (T&E)  Level  of  Effort  (LoE)  proportional  to  the  ACAT  level?"  The 
secondary  questions  arc  “How  docs  program  oversight  by  the  Director,  Operational  Test 
and  Evaluation  or  the  Army  drive  the  T&E  LoE?"  and  "Are  Chemical/Biological  (C/B) 
programs  a  special  case  for  T&E  LoE?" 

The  analysis  is  presented  in  three  parts,  data  scope  and  limitations,  analysis 
approach,  and  data  analysis.  Understanding  the  limitations  and  scope  of  the  data  set  is 
critical  to  the  development  of  a  dcfcndablc  analysis  approach  and  to  accurate 
interpretation  of  the  results  obtained.  Similarly,  the  analysis  must  be  performed  within 
the  context  of  the  data  set  and  the  results  obtained  qualified  based  upon  the  data  and 
analysis  limitations.  The  analysis  presented  in  this  section  is  framed  to  ensure  that  the 
context  of  the  baseline  data  is  maintained. 

B.  LIMITATIONS  AND  SCOPE 

The  principal  data  source  for  this  effort  was  the  TEMP  for  each  program.  Other 
program  documents,  including  performance  specifications,  OSD  budget  and  program 
oversight  data.  Single  Acquisition  Management  Plan  (SAMPs),  APBs,  and  requirements 
documents  were  used  to  fill  TEMP  data  gaps  and  to  help  answer  the  secondary  research 
questions. 

During  data  collection  efforts,  it  became  clear  that  the  variability  in  the  level  of 

detail  and  the  types  of  information  included  in  CBD  TEMPs  would  also  limit  the  scope  of 

this  analysis  effort.  Although  these  TEMPs  generally  included  an  Integrated  Program 

Summary  (IPS)  and  cost  data,  the  level  of  detail  and  consistency  of  data  program-to- 

program  did  not  support  a  comparative  analysis  of  test  schedules  or  of  program  R&D. 

test,  and  production  costs.  A  comparison  of  program  schedules  and  budgets,  taken  from 

the  OSD(C)  website,  also  did  not  generate  data  sufficient  to  perform  a  test  costs 
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comparison  of  CBD  programs.  This  lack  of  test-specific  data  rc-scoped  the  analysis  to 
the  technical  and  operational  requirements  and  testing  listed  in  each  TEMP  and  to  the 
documentation  scope  of  each  TEMP. 

Another  limitation  of  the  analysis  effort  was  the  difficulty  in  gaining  access  to 
non-CBD  TEMPs.  The  team  was  only  able  to  obtain  copies  of  TEMPs  for  two  Army 
programs  outside  of  the  CBDP.  Stryker  and  Future  Combat  Systems  (FCS).  Both  of 
these  programs  arc  ACAT  I-D.  Therefore,  analysis  in  support  of  the  secondary  research 
question,  "Are  Chemical/Biological  programs  a  special  ease  for  T&E  LoE?”  is 
constrained  by  this  limited  data  set. 

C.  APPROACH 

The  answer  to  the  primary  research  question  was  developed  through  an  analysis 
of  the  system  requirements,  documentation  scope,  and  test  scope.  The  secondary 
research  questions  were  answered  by  further  dividing  the  existing  data  into  smaller 
subsets,  based  on  oversight  and  CBD  program  status. 

1.  System  Requirements 

The  initial  approach  to  data  analysis  was  to  perform  a  comparison  of  the  system- 
level  requirements  in  each  TEMP  to  the  scope  of  the  resultant  testing,  sorted  by  ACAT. 
Requirements  were  sorted  into  two  main  categories.  Operational  Effectiveness  <OE),  and 
Operational  Suitability  (OS).  Each  requirement  was  further  categorized  to  a  functional 
area  (c.g.,  operation,  interoperability,  environmental,  mobility,  primary  technical 
performance  characteristic  (such  as  detection),  and  logistics).  This  analysis  approach 
allowed  a  comparison  of  detection  systems,  but  broke  down  for  other  types  of  CBD 
programs,  such  as  vehicles,  shelters,  and  individual  protection  due  to  the  lack  of 
consistent  requirement  categorization  and  detail  levels  in  TEMPs  from  different 
functional  areas. 

A  mathematical  approach  was  developed  to  normalize  data  across  TEMPs  after  it 
was  determined  that  direct  comparisons  of  OE  and  OS  would  not  be  possible.  The 
change  to  a  mathematical  analysis  enabled  a  direct  comparison  of  programs  without  the 
need  for  a  requirement-to-requirement  crosswalk.  This  requirements  analysis  technique 
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is  based  on  calculation  of  a  program-specific  quantified  requirements  ratio  using  the 
MOE,  MOS,  and  CTPs,  followed  by  an  ACAT-bascd  summarization.  A  similar  ratio  was 
also  calculated  for  COIs.  The  analysis  results  were  then  reviewed  to  determine  what,  if 
any,  ACAT-spccific  trends  were  apparent. 

2.  Documentation  Scope 

Per  DA  Pamphlet  73-1,  “the  target  size  of  a  TEMP  should  be  approximately  30 
pages,  including  pages  for  figures,  tables,  matrices,  and  so  forth.  Although  annexes  and 
attachments  arc  excluded  from  the  30-pagc  limit,  their  size  should  be  kept  to  a 
minimum."  [1 1,  p.  15J  Likewise,  the  Navy’s  Operational  Test  Director's  Guide  suggests, 
“a  TEMP  is  generally  limited  to  30  pages."  [16,  p.  5-3]  Since  the  Army  and  Navy 
provides  explicit  guidance  for  TEMP  page  counts,  the  documentation  scope  analysis 
consists  of  tallying  the  sections  1  -  V  and  total  page  counts  for  the  TEMPS,  then 
comparing  the  averages  for  each  ACAT.  Secondarily,  the  effects  of  test  program 
oversight  and  a  time-based  analysis  of  the  scope  of  ACAT  III  TEMPs  will  be 
investigated. 

3.  Test  Scope 

The  scope  of  testing  involves  many  factors;  including  test  sites  used,  types  of 
tests,  funding  used,  and  time  and  manpower  required  for  testing.  The  analysis  of  test 
scope  will  focus  on  the  involvement  of  test  conduct  and  evaluation  organizations  and  the 
types  of  tests  each  program  listed  (c.g.,  MIL-STD-810  environmental  tests)  to  determine 
whether  there  are  any  ACAT  level  biases  to  test  scope. 

D.  DATA  ANALYSIS 

1.  System  Requirements 

It  was  noticed  during  data  collection  that  some  of  the  requirements  in  each  TEMP 
were  more  tcclmically-focuscd,  or  quantifiable,  than  others.  The  quantifiable  versus  non- 
quantifiable  nature  of  each  requirement  was  captured  as  a  secondary  eharaetcristic  of  the 
data. 
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The  simple  set  of  questions  developed  to  categorize  whether  a  requirement  is 
quantifiable  arc, 

•  Can  a  determination  of  whether  the  system  meets  the  requirement  be  made 
by  simple  observation  or  as  an  observable  byproduct  of  a  test  for  another 
characteristic?  If  so,  the  requirement  is  not  quantifiable. 

•  Docs  determination  of  whether  the  system  meets  the  requirement  require  a 
formal  nontrivial  test?  If  so,  the  requirement  is  quantifiable. 

•  Does  the  requirement  include  performance  parameters  that  only  require  a 
yes/no  or  go, 'no-go  answer?  If  so,  the  requirement  is  not  quantifiable. 

Typical  formats  for  quantifiable  requirements  arc, 

•  MOE,  Probability  of  Detection  (Pd)  =  92%  for  point  concentration 

•  MOE,  Accomplish  OMS/MP  following  a  HEMP  event 

•  MOS,  MTBFA  >  246  hr 

•  MOS,  Minimum  of  1000  mean  miles  between  critical  vehicle  failures 

•  CTP,  Collective  pressure  system  with  2.5  1VVG  overpressure  at  90  CFM 

•  CTP,  Meets  EM  I  requirements  of  M I L-STD-46 1 ,  test  RS 1 03 
Non-quantifiablc  requirements  are  more  general  in  nature,  such  as, 

•  MOE,  Able  to  be  powered  by  rechargeable  and  non-rcchargeablc 
batteries 

•  MOE,  Shall  be  worn  inside  combat  footwear 

•  MOS,  Includes  a  confidence  checker 

•  MOS,  Crew  size  of  not  more  than  4  crew  members 

•  CTP,  HMMWV  and  M 1 1 3  mounted  ability 

•  CTP,  Equipped  with  run-flat  tires 

The  total  quantifiable  requirements  per  TEMP  arc  listed  in  Table  1  for  CTPs  and 
in  Table  2,  for  the  combination  of  MOE  and  MOS.  Two  factors  contributed  to  the 
merging  of  MOE  with  MOS  in  the  analysis  (where  MOS  includes  suitability  and 
survivability);  the  lack  of  discrimination  between  MOE  and  MOS  within  many  of  the 
TEMPs  and  the  operational  focus  of  both  requirements  when  compared  to  the  technically 
oriented  CTPs. 

Determination  of  whether  a  requirement  was  quantifiable  or  non-quantifiablc 
neither  directly  allowed  a  meaningful  comparison  of  programs,  nor  did  it  provide  insight 
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into  any  ACAT-specific  relationships.  Since  the  number  of  quantifiable  and  total 
requirements  in  the  form  of  MOE,  MOS,  and  CTPs  varied  widely  over  the  twenty  three 
programs  analyzed,  a  straightforward  comparison  based  upon  numbers  of  requirements 
did  not  yield  any  useful  relationships.  A  comparison  of  individual  requirements  did  not 
prove  feasible,  also  due  to  the  varied  nature  of  system-specific  requirements,  even  within 
the  CBDP. 

Only  with  the  development  of  a  program-specific  ratio  of  quantifiable  to  total 
requirements,  was  it  possible  to  compare  programs  in  an  analytical  manner.  The  ratio 
developed  is  called  the  CTP/MOE-MOS  ratio,  or  more  succinctly  the  C/M-M  ratio. 

Calculation  of  the  C/M-M  ratio  requires  the  preliminary  calculation  of  the 
quantifiable  MOE  and  MOS  ratio,  and  the  calculation  of  the  quantifiable  CTP  ratio. 
Once  these  values  arc  defined,  the  C/M-M  ratio  is  calculated  by  dividing  the  quantifiable 
CTP  ratio  by  the  quantifiable  MOE  and  MOS  ratio.  The  formula  is. 

Quantifiable  CTP  Ratio 

C/M-M  Ratio  = - 

Quantifiable  MOE-MOS  Ratio 


where. 

Number  of  Quantifiable  CTPs 

Quantifiable  CTP  Ratio  = - 

Total  Number  of  CTPs 


Quantifiable  MOE+MOS 

Quantifiable  MOE-MOS  Ratio  = - 

Total  Number  of  MOE+MOS 


The  program-specific  C/M-M  ratios  arc  listed  in  Table  13,  Calculated  System 
Requirements  Ratios,  and  graphed  in  Figure  7,  Quantified  C/M-M  Ratio  for  All 
Programs.  Table  13  also  includes  a  COI-based  ratio,  which  is  defined  as  the  number  of 
COIs  divided  by  the  C/M-M  ratio.  The  formula  for  the  COI/C/M-M  ratio  is. 
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Number  of  COIs 


COI/C/M-M  Ratio  = - 

C/M-M  Ratio 


ACAT 

PROGRAM 

Program  System  Requirements 

Ratios 

Percent 

Quantifiable 

MOE/MOS 

Percent 

Quantifiable 

CTPs 

C/M-M 

Ratio 

COI/ 

C/M-M 

Ratio 

FCS 

2.9 

m 

16.2 

m 

I 

NBCRV 

13.2 

4.9 

Us 

STRYKER 

57.9 

2.3 

3.89 

JBPDS 

m 

42.9 

1.6 

2.57 

II 

JCAD 

49.3 

EH 

1.6 

mm 

JSLNBCRS 

21.1 

Wsm  . 

1.9 

n 

ACADA 

74.3 

76.5 

1.0 

2.91 

ALSI-I 

60.0 

75.0 

m  ' 

3.20 

BIDS 

50.0 

33.3 

4.50 

CBPSS 

ps  : 

41.7 

K9 

wm 

FOX 

Hi  : 

77.8 

JBAIDS 

66.7 

i.i 

1.87 

JBSDS 

50.0 

1.3 

2.25 

JPACE 

31.8 

1.6 

2.55 

III 

JSGPM 

29.1 

1.3 

2.24 

JSLIST-GLOVE 

52.0 

59.3 

1.1 

2.63 

JSLSCAD 

64.7 

72.4 

1.1 

JSTDS 

37.3 

49.2 

1.3 

LVOSS 

38.1 

36.4 

1.0 

2.10 

M45 

61.5 

83.3 

1.4 

2.95 

M56 

iH  : 

100.0 

2.7 

0.74 

M6 

in  ■ 

42.9 

1.2 

1.73 

SHIPBRD-CPS 

83.3 

81.8 

2.04 

Table  13.  Program-Level  System  Requirements  Ratios 
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Both  ratios  were  developed  to  provide  a  measure  of  the  technical  orientation  of  the 
operational  requirements  for  a  system.  These  ratios  also  introduce  a  level  of  abstraction 
to  the  analysis,  which  proved  essential  to  performing  meaningful  comparisons  of  systems 
at  the  AC  AT  level. 


CU  M  Ratio  -  b*  Program 


ElACATl  BACAT  I  DACAT  lll\O0l  SACAT  III  (New) 


Figure  7.  Quantified  C/M-M  Ratio  for  All  Programs 


COI/C/M-M  Ratio  by  ACAT 


- I _  -  J _ IBtta _ l— I _ f.V.V.I _ ...... _ _ 

ACAT  I  ACAT  I  ACAT  II  ACAT  III  ACAT  III  ACAT  III  ACAT  III 
<FCS)  (Stryker)  <AII)  <Old>  (New)  <  Oversight) 


Figure  8.  COl/C/M-M  Ratios  by  ACAT 
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Systems  Requirements  Ratio  Summary  J 

ACAT 

Mean 

C/M-M 

Ratio 

Standard 

Deviation 

of 

C/M-M 

Ratio 

Mean  + 
Standard 
Deviation 

Mean  — 

Standard 

Deviation 

Standard 

Deviation 

as  a 

Percent  of 
C/M-M 
Ratio 

Mean 

Mean 

COI/ 

C/M-M 

Ratio 

I 

(All) 

7.78 

7.36 

15.14 

0.41 

94.7 

2.31 

1 

(Stryker) 

3.59 

1.80 

5.39 

1.79 

50.2 

3.28 

II 

1.70 

0.21 

1.91 

1.48 

12.6 

2.00 

III 

(All) 

1.22 

0.44 

1.66 

0.78 

35.9 

2.74 

III 

(Old) 

1.18 

0.63 

1.81 

0.55 

53.3 

2.90 

III 

(New) 

1.26 

0.26 

1.53 

1.00 

20.9 

2.57 

III 

(Over¬ 

sight) 

1.17 

0.14 

1.31 

1.04 

11.9 

2.56 

Tabic  14.  System  Requirements  Ratio  Summary  by  ACAT 
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C/M-M  Ratio 


C/M-M  Mean  -  Variability  by  ACAT 


ACATI  ACAT  I  ACAT  II  ACAT  III  ACAT  III  ACAT  III  ACAT  III 
<AII)  <Sliykei)  CAI>  <Old>  (flew)  <Oveis»ght> 


ACAT  Levels 

Figure  9.  Mean  C/M-M  Values  by  ACAT  with  Standard  Deviations 
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After  the  requirements-based  ratios  were  calculated  at  the  program  level,  a 
summary  at  the  ACAT  level  was  performed.  A  review  of  the  results  for  ACAT  1,  II.  and 
111  programs  showed  a  high  level  of  variability,  as  evidenced  by  the  large  standard 
deviations  on  the  summarized  data  in  Table  14,  System  Requirements  Ratio  Summary  by 
ACAT.  The  standard  deviation  for  all  ACAT  I  programs  (All  Pgms)  was  94.7  percent  of 
the  mean  value  for  the  C/M-M  ratio  and  the  standard  deviation  for  ACAT  111  (all 
programs)  was  35.9  percent  of  the  mean  value  for  the  C/M-M  ratio.  A  second  look  at  the 
data  showed  that  for  ACAT  I  systems  the  FCS  data  was  significantly  outside  of  the 
Stryker  vehicles  data  range,  and  for  ACAT  III  programs  whose  TEMPs  were  dated  more 
than  five  years  ago,  appeared  to  dilTcr  significantly  from  more  recent  TEMPs.  Therefore, 
additional  analysis  was  performed  to  look  at  what  effects  these  data  differences  may  have 
at  the  ACAT  level.  As  a  result  of  this  finding.  Table  14  includes  multiple  categories 
within  ACAT  I  and  ACAT  HI  summarized  data.  Additionally,  a  category  for  ACAT  III 
oversight  programs  was  added  to  help  answer  the  research  question  related  to  effects  of 
oversight  on  programs. 

The  calculations  required  to  generate  the  data  in  Table  14  arc, 

n 

£  [C/M-M  Ratios] 

1 

Mean  C/M-M  ratio  = - 

n 


where, 

n=number  of  ratios  in  the  data  set 


Standaid  Deviation  (CJ)  = 


/Z(x-x)2 

V  (n-1) 


where,  x  =  sample  mean  average,  n  =  sample  size 
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Mean  +  CT  =  Mean  C/M-M  ratio  +  (7  of  C/M-M  ratios  used  to  calculate  the  mean 
Mean  -  CT  =  Mean  C/M-M  ratio  -  <5  of  C/M-M  ratios  used  to  calculate  the  mean 


O  as  Percent  of  Mean  =  I  — -  _  I  X  100 

\Mean  C/M-M  Ratio  j 

n 

£  [COl/CM-M  Ratios) 

1 

Mean  COl/C/M-M  ratio  = - 

n 

where,  n  =  number  of  ratios  in  the  data  set 

The  C/M-M  ratio  data  was  also  compared,  on  a  frequency  of  occurrence  basis,  to 
determine  if  the  ratios  followed  any  disccmablc  pattern,  cither  overall  or  by  ACAT  level. 
Table  15,  Frequency  of  C/M-M  Value  Occurrence  by  ACAT,  lists  each  C/M-M  value 
obtained  versus  the  total  number  of  occurrences  per  ACAT.  This  data  is  graphed  in 
Figure  8,  Frequency  Distribution  of  C/M-M  Values  by  ACAT.  The  Figure  shows  data 
that  appears  to  fall  into  a  normal  distribution  with  a  bell-shaped  curve  for  ACAT  II  and 
ACAT  III  programs.  The  broad  range  and  limited  data  set,  three  data  points,  for  ACAT  1 
programs  showed  no  discernablc  pattern  in  relation  to  the  ACAT  II  and  III  programs. 
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C/M-M 

Ratio 

Frequency  of  C/M-M  Value  Occurrence 

ACAT  I 

ACAT  II 

ACAT  III 
(Old) 

ACAT  III 
(New) 

0.7 

1 

0.8 

1 

1.0 

4 

1.1 

3 

1.2 

I 

1.3 

4 

1.4 

1 

1.6 

2 

1 

1.9 

I 

2.3 

1 

2.7 

1 

4.9 

1 

16.2 

1 

Table  1 5.  Frequency  of  C/M-M  Value  Occurrence  by  ACAT 
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The  requirements  analysis  results  have  demonstrated  that  the  C/M-M  ratio  does 
differentiate  between  programs  based  on  ACAT.  As  a  whole,  the  ACAT  III  programs 
have  a  lower  average  value  for  the  C/M-M  ratio.  The  ACAT  II  programs  fit  into  the  high 
end  of  a  normal  distribution  of  frequency  of  C/M-M  ratios  when  combined  with  the 
ACAT  III  programs.  The  C/M-M  ratios  for  ACAT  I  programs  arc  highly  variable  and  arc 
typically  much  higher  than  the  values  found  for  ACAT  II  and  111  programs.  A 
comparison  of  the  quantity  of  program  COIs  to  the  C/M-M  ratio  does  not  demonstrate 
any  disccmablc  difference  between  ACAT  levels. 


2.  Documentation  Scope 


The  second  phase  of  analysis  in  the  investigation  of  the  test-related  LoE  for 
programs  involves  determining  the  documentation  burden  posed  by  the  program  TEMPs. 
The  overall  size  of  each  TEMP  and  the  size  of  sections  I  to  V  was  gathered  and 
summarized  at  the  ACAT  level  and  then  compared  to  identify  any  trends.  The  page 
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count  data  from  Tabic  3,  COls  and  TEMP  Page  Counts,  and  the  date  and  oversight  status 
from  Table  4,  TEMP  Data  and  Oversight  Requirements,  are  summarized  in  Table  16, 
TEMP  Documentation  Size  Summary  by  ACAT.  The  standard  deviation  of  the  TEMP 
Sections  I-V  page  counts  was  also  calculated  and  listed  in  Table  16.  The  FCS  and 
Stryker  ACAT  I  programs  arc  listed  separately  in  Table  16  due  to  the  extreme  differences 
in  documentation  scope  for  these  vehicle-based  programs.  Finally,  Table  16  presents  the 
relationship  between  the  mean  page  counts  and  the  DA  Pamphlet  73-1  recommendation 
of  a  30-page  maximum  for  TEMP  Sections  I-V.  The  data  presented  in  Table  16  is 
graphed  in  Figure  1 1,  TEMP  Page  Count  Comparison  by  ACAT. 


Figure  1 1 .  TEMP  Page  Count  Comparison  by  ACAT 
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TEMP  Documentation  Size  Summary 

ACAT 

TEMP 
Section 
I-V  No. 
Pages 

Standard 
Deviation 
of  Mean 

TEMP 

Sec.  I-V  No. 
Pages 

Standard 
Deviation, 
as  a  Percent 

of  TEMP 
Sec.  I-V 
Mean  No. 
Pages 

TEMP 

Sec.  I-V 
Pages,  as  a 
Percent 
of  30  page 
DA  PAM 
Guidance 

Mean 

TEMP  Total 
No.  Pages 

m 

93 

N/A 

N/A 

310% 

482 

i 

(Strvker)+ 

58 

0.71 

1.2% 

192% 

106 

II 

75 

17.8 

23.8% 

249% 

123 

III 

(All) 

39 

17.8 

45.8% 

129% 

64 

III 

(Old) 

27 

9.7 

35.7% 

90% 

38 

III 

(New) 

52 

15.6 

30.0% 

173% 

90 

III 

(Over- 

sight) 

51 

16.8 

33.1% 

169% 

92 

Tabic  16.  TEMP  Documentation  Size  Summary  by  AC  AT 


*  (FCS  data  is  from  a  single  program  TEMP,  +  Stryker  data  is  from  two  program 
TEMPs) 

Multiple  findings  arc  present  in  the  results  of  the  page  count  analysis.  First,  only 
the  older  ACAT  III  program  group  met  the  30-page  recommendation  for  TEMP  Sections 
I-V.  Second,  the  TEMP  documentation  size  for  new  ACAT  III  programs  docs  not  appear 
to  depend  upon  program  oversight  status.  Third,  the  documentation  size  for  ACAT  III 
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programs  is  comparable  to  that  of  the  AC  AT  I  Stryker  programs.  Fourth,  the  ACAT  II 
programs’  TEMP  documentation  size  was  higher  on  average  than  all  other  programs, 
with  one  exception,  the  ACAT  I  FCS  program. 

3.  Test  Scope 

The  final  phase  of  analysis  in  the  investigation  of  the  test-related  LoE  for 
programs  involves  reviewing  the  scope  of  test  related  efforts  documented  in  the  program 
TEMPs.  The  data  contained  in  Tables  5  through  12  was  analyzed  to  determine  whether  a 
test-specific  relationship  exists  between  a  program's  ACAT,  joint  designation,  or 
oversight  status.  The  number  of  occurrences  of  each  item  was  summed  over  the  data 
category  (c.g.,  ACAT  II  programs;  which  include  the  JBPDS.  JCAD.  and  JSLNBCRS) 
then  the  mean  calculated  by  dividing  the  total  occurrences  by  the  number  of  programs  in 
the  data  set  (c.g.,  three)  to  determine  the  value. 

Table  17,  Test  Scope  Summary  by  ACAT,  shows  the  results  using  the  same 
ACAT-based  categories  as  the  requirements  and  documentation  analysis  sections.  Table 
18,  Test  Scope  Summary  by  Joint  and  Oversight  Status,  presents  an  ACAT  neutral  view 
of  how  a  program’s  jointness  or  T&E  oversight  status  may  impact  the  LoE.  Both  tables 
include  columns  for  the  number  of  programs  in  each  category,  the  mean  number  of  test 
evaluation  organizations  used,  the  mean  number  of  Army  and  other  test  organizations 
used,  and  the  total  of  test  organizations.  Table  17  is  organized  using  the  same  ACAT- 
based  breakout  as  in  the  requirements  and  documentation  analysis  to  enable  one-to-one 
comparisons  of  the  results.  Table  18  is  organized  to  provide  insight  into  joint  versus 
Army-only  programs  and  T&E  oversight  to  see  if  these  attributes  drive  test  efforts. 

The  test  scope  evaluation,  while  limited  to  a  comparison  of  the  numbers  of 
organizations  involved  in  test  evaluation  and  test  conduct,  provides  some  insight  into  the 
effects  of  ACAT  level,  joint  designation,  and  oversight  status.  A  review  of  the  number  of 
test  evaluation  organizations  involved  in  programs,  as  graphed  in  Figure  12  from  data  in 
column  3  of  Tables  17  and  18,  shows  several  relationships. 


68 


Test 

Scope  Summary  - 1 

Number 

Mean  No.  of 

Mean  No.  of 

Mean  Total 

ACAT 

of 

Army  Test 

Other  Service  + 

of  all 

Programs 

Orgs. 

Commercial  Test 

Test  Orgs. 

ETS1 

Orgs. 

I 

3 

1.0 

9.0 

0 

9.0 

II 

3 

1.3 

5.7 

5.7 

1 1.4 

III 

17 

1.9 

5.2 

1.4 

6.6 

(All) 

III 

9 

l.l 

6.1 

0.9 

7.0 

(Old) 

III 

8 

2.4 

4.1 

2.0 

6.1 

(New) 

III 

3 

1.7 

3.3 

0.7 

4.0 

(Oversight) 

Tabic  1 7.  Test  Scope  Summary  by  ACAT 


ACAT 

Independent 

Program 

Type 

Test  Scope  Summary  -  II 

Number 

of 

Programs 

Mean 

No.  of 
Evaluation 
Orgs. 

Mean  No. 
of  Army 
Test  Orgs. 

Mean  No.  of 

Other  Service 
+  Commercial 
Test  Orgs. 

Mean 
Total  of 

all 

Test 

Orgs. 

Joint 

10 

2.2 

4.4 

1.9 

6.3 

13 

1.1 

6.8 

0.6 

7.4 

9 

1.3 

6.0 

2.1 

8.1 

Non-Oversight 

14 

1.7 

5.6 

1.6 

7.2 

Table  1 8.  Test  Scope  Summary  by  Joint  and  Oversight  Designation 
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Figure  12.  Test  Evaluation  Organization  Involvement 


Figure  1 3.  Number  of  Test  Organizations/Sites  Used 
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ACAT  I,  ACAT  III  (Old),  and  Army-Only  programs  utilized,  on  average,  only 
one  evaluation  organization.  This  result  makes  intuitive  sense,  as  all  three  of  these 
program  types  arc  Army-only  or  non-joint  designated  programs.  For  these  programs 
AEC  filled  the  test  evaluation  role. 

ACAT  III  (New)  and  Joint  Designated  programs  have  the  highest  average  number 
of  involved  evaluation  organizations.  It  also  was  expected  that  these  two  categories  of 
programs  would  have  similar  values,  since  seven  of  the  eight  ACAT  III  (New)  programs 
evaluated  were  joint  programs.  The  other  joint  programs  analyzed  were  the  three  ACAT 
II  programs,  which  had  a  lower  value  than  the  ACAT  Ill’s,  which  pulled  the  joint 
designated  average  down  slightly.  This  category  had  more  than  twice  the  average 
number  of  evaluators  as  the  Army-only  and  ACAT  I  programs. 

ACAT  III  (Oversight)  and  all  non-oversight  programs  had  the  same  value  of  1.7 
organizations,  and  ACAT  II  and  all  oversight  programs  had  a  lesser,  but  identical  value 
of  1.3  organizations.  These  results  demonstrate  that  program  oversight  generally  reduces 
the  number  of  test  evaluators  involved  in  a  program,  and  that  ACAT  III  oversight 
programs  have  a  similar  evaluator  involvement  as  most  programs  without  oversight. 

Figure  12  is  split,  with  the  left  side  showing  an  ACAT-based  comparison,  and  the 
right  side  showing  the  non-ACAT  factors  that  may  contribute  to  differences  between 
programs.  The  analysis  of  test  evaluation  organizations  depicted  in  Figure  12  shows 
nearly  identical  means  and  standard  deviations,  regardless  of  the  basis  of  comparison. 
The  means  and  standard  deviations  are  1 .57  and  0.54  for  ACAT-based,  and  1 .58  and  0.49 
for  non- ACAT-based  views.  Therefore,  the  ACAT  and  the  other  factors  evaluated  arc 
equivalent  in  their  intluencc  on  the  results. 

The  scope  of  T&E  efforts  related  to  numbers  of  test  organizations,  and  test  sites, 
used  is  depicted  in  Figure  13,  which  is  also  split  to  allow  ACAT  and  non-ACAT 
comparisons.  Most  programs  fall  within  a  one  standard  deviation  band  of  total  test 
organization  and  site  usage.  As  would  be  expected.  Army-only  programs  tended  to 
primarily  use  Army  test  sites.  Only  ACAT  II  and  ACAT  III  (Oversight)  programs  fell 
outside  of  the  one  standard  deviation  band.  ACAT  II  programs  used  as  many  non- Army 
test  organizations  and  sites  as  Army  ones,  while  ACAT  III  (Oversight)  programs  arc 
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significantly  below  this  band.  Interestingly,  the  category  that  used  the  fewest 
organizations  and  sites  was  ACAT  III  (Oversight). 

The  non-ACAT  comparison  of  test  sites  in  Figure  13  shows  a  much  lower 
standard  deviation  than  the  ACAT-based  results.  The  means  and  standard  deviations  arc 
7.35  and  2.55  for  ACAT-based.  and  7.25  and  0.74  for  non- ACAT-based  views.  This 
demonstrates  that  the  ACAT  is  the  primary  factor  for  differences  in  the  number  of  test 
organizations  and  sites  used. 

E.  CHAPTER  SUMMARY 

The  analysis  of  the  data  shown  in  Chapter  III  was  performed  in  three  parts, 
system  requirements,  documentation  scope,  and  test  scope.  The  analysis  was  performed 
on  twenty-three  programs,  primarily  using  data  extracted  from  TEMPs,  to  answer  the 
primary  and  secondary  research  questions. 

The  requirements  analysis  has  shown  that  ACAT  III  CB  programs  have  the  most 
technically-oriented,  quantifiable,  MOE  and  MOS.  ACAT  II  CB  program  MOE  and 
MOS  arc  only  slightly  less  quantifiable  than  those  of  ACAT  III  CB  programs.  The 
ACAT  I  programs  evaluated  had  the  least  technically  focused  MOE  and  MOS 
requirements. 

The  documentation  scope  analysis  determined  that  none  of  the  current  CB  and 
Army  programs  met  the  DA  Pamphlet  73-1  guidance  for  Sections  I-V  TEMP  pages. 
Also,  TEMPs  for  an  average  ACAT  III  CB  program  arc  equivalent  in  volume  to  the 
ACAT  I  Stryker  programs.  The  impact  of  oversight  could  only  be  evaluated  for  ACAT 
III  programs,  where  no  noticeable  differences  in  TEMP  scope  could  be  discerned. 

The  test-related  analysis  determined  that  the  average  number  of  test  evaluation 
organizations  used  was  not  directly  related  to  the  ACAT  level.  There  were  two  outliers 
in  the  analysis,  one  was  ACAT  I  programs  and  the  other  was  joint-designated  programs. 
ACAT  I  programs  used  only  one  evaluation  organization,  while  joint-designated 
programs  averaged  more  than  twice  that  number. 

The  test  related  analysis  also  did  not  find  any  noticeable  trends  between  for  the 
joint  designated.  Army-only,  oversight,  or  non-oversight  categories.  However.  ACAT  III 
(Oversight)  programs  and  ACAT  II  programs  were  statistically  outside  of  the  norm  for 
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average  number  of  test  organizations  and  sites  used.  The  extensive  use  of  non-Army  test 
facilities  drove  the  ACAT  II  programs  significantly  higher  than  other  program  categories, 
while  the  reason  that  ACAT  III  (Oversight)  programs  were  significantly  lower  than  other 
programs  was  not  directly  evident. 

In  summary,  the  analysis  has  determined  that, 

•  ACAT  II  programs  arc  the  most  burdened  CB  programs;  combining 
highly-technical  MOEs  and  MOSs,  joint-designation,  the  highest  TEMP 
documentation  burden,  and  the  broadest  test  scope. 

•  ACAT  I  programs  are  the  least  burdened  by  technical  MOE  and  MOS 
requirements,  while  ACAT  III  programs  are  the  most  burdened. 

•  Oversight  appears  to  have  a  neutral  to  positive  impact  on  ACAT  III 
programs. 

•  The  TEMP  documentation  burden  for  CB  programs  is  not  related  to 
ACAT. 

•  Joint-designation  noticeably  increases  the  test  and  documentation  burden. 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  INTRODUCTION 

This  chapter  presents  the  conclusions  reached  regarding  the  primary  and 
secondary  research  questions  of  this  study.  The  conclusions  are  derived  from  the  analysis 
conducted  in  Chapter  IV,  using  the  background  information  presented  in  Chapter  II,  the 
data  presented  in  Chapter  III.  and  the  overall  scope  of  the  study.  Recommendations  for 
further  research  arc  presented  in  order  to  focus  potential  follow-on  investigations  toward 
areas  that  will  help  complete  the  overall  picture  of  ACAT-rclated  program  test  burden.  A 
recommendation  related  to  TEMP  implementation  is  also  presented  to  foster  discussions 
of  T&E  management  processes. 

B.  CONCLUSIONS 

Level  of  Effort  is  a  key  phrase  in  all  three  study  research  questions.  This  research 
study  evaluated  the  LoE  for  T&E  within  a  program  using  three  factors, 

•  How  technically  focused  the  operationally  evaluation  parameters,  MOE 
and  MOS,  were  in  relation  to  the  technical  evaluation  parameters,  CTPs. 

•  The  T&E  documentation  burden  to  a  program,  based  on  the  TEMP  size. 

•  The  number  of  T&E  organizations  involved  in  the  conduct  and  evaluation 
of  program  tests. 

I.  Primary  Research  Question  -  T&E  LoE  and  ACAT 

The  primary  research  question  is,  "Is  the  Test  and  Evaluation  (T&E)  Level  of 
Effort  (LoE)  proportional  to  the  ACAT  level?" 

The  T&E  LoE  is  not  proportional  to  the  ACAT  level. 

Each  ACAT  level  scored  best  in  one  or  more,  and  worst  in  one  or  more,  of  the 
categories  evaluated.  The  T&E  LoE  was  analyzed  using  the  three  factors  listed  above. 
The  ACAT  1  programs  evaluated  had  the  best  differentiated  technical  and  operational 
requirements  and  used  the  least  number  of  evaluation  organizations.  ACAT  II  programs 
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had  technical ly-based  operational  requirements,  a  high  LoE  for  TEMP  documentation, 
and  used  the  most  test  sites.  ACAT  III  programs  had  the  most  technically-based 
operational  requirements,  similar  TEMP  documentation  to  the  other  ACATs,  with  the 
most  evaluator  involvement,  but  used  less  test  sites  overall.  These  conflicting  results 
cemented  the  conclusion  that  ACAT  level  and  T&E  LoE  do  not  correlate  directly. 

2.  Secondary  Research  Question  -  Oversight 

The  first  secondary  research  question  is  “How  docs  program  oversight  by  the 
Director  Operational  Test  and  Evaluation  or  the  Army  drive  the  T&E  LoE?" 

T&E  oversight  had  a  neutral  to  positive  effect  on  ACAT  111  CB  programs. 

Oversight  comparisons  could  only  be  performed  for  ACAT  III  programs,  as  there 
were  no  non-oversight  ACAT  1  and  II  programs  in  this  study.  ACAT  III  oversight 
programs  showed  improvements  over  non-oversight  programs  by  using  fewer  test  sites 
and  through  less  variability  in  the  C/M-M  ratio  within  the  TEMP.  The  overall  C/M-M 
ratio,  the  TEMP  LoE,  and  the  number  of  test  evaluators  were  all  comparable  to  non- 
oversight  ACAT  III  programs. 

3.  Secondary  Research  Question  -  CB  Programs 

The  second  secondary  research  question  is  “Are  Chcmical/Biological  (C.'B) 
programs  a  special  case  for  T&E  LoE? 

An  answer  to  this  question  could  not  directly  be  determined. 

Since  only  three  of  the  programs  studied  were  not  part  of  the  CBDP,  and  they 
were  all  ACAT  I  programs,  there  was  insufficient  data  to  determine  whether  CB 
programs  arc  unique  within  DoD  for  their  T&E  LoE.  Data  related  to  the  joint  nature  of 
most  CB  programs  was  gathered  and  analyzed,  but  this  data  was  not  broad  enough  to 
answer  this  overarching  question. 
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c. 


RECOMMENDATIONS 


1.  Areas  of  Further  Study 

Although  this  research  study  was  able  to  find  useful  relationships  between 
programs  of  different  ACATs,  including  the  development  of  a  new  requirements 
quantification  ratio,  it  was  constrained  by  the  limitations  of  the  available  data.  Specific 
additional  study  areas  that  have  the  potential  to  significantly  increase  the  overall 
understanding  of  how  T&E  is  performed  on  different  ACAT-lcvcl  programs  are, 

•  An  analysis  of  acquisition  programs  in  other  commodity  areas  within 
DoD,  using  this  study's  methodology.  This  research  will  help  determine 
whether  CB  programs  are  typical  or  unique  within  DoD. 

•  An  analysis  of  program  T&E  funding  to  overall  program  funding  on  an 
ACAT  and  non-ACAT  basis,  for  both  Research,  Development,  Test  & 
Evaluation  (RDT&E)  and  Procurement  funds  for  CB  programs.  This 
research  will  enable  a  requirements  versus  cost  analysis,  which  was  not 
able  to  be  completed  with  existing  TEMP  cost  data. 

•  An  analysis  of  the  number  and  types  of  tests  conducted,  based  upon 
ACAT  and  non-ACAT  factors.  This  information  will  provide  the  final 
details  necessary  to  determine  what,  if  any,  factors  drive  detailed  program 
tests. 

2.  T&E  Program  Implementation 

One  area  that  should  be  reevaluated  is  the  Army's  recommendation  for  no  more 
than  30  pages  for  Sections  I-V  of  a  TEMP.  Only  the  older  ACAT  111  programs  studied 
met  this  goal,  and  current  Army/CBDP  TEMPs  appear  to  have  normalized  on  a  50-60 
page  average.  Since  none  of  the  current  acquisition  programs  studied  met  this 
requirement,  regardless  of  ACAT,  the  time  may  be  right  to  rethink  this  guidance. 

D.  SUMMARY 

This  research  study  was  undertaken  to  investigate  the  validity  of  the  common 
belief  that  smaller  programs  endure  a  disproportionately  heavy  T&E  burden  relative  to 
larger  programs.  In  general,  this  notion  was  proven  true.  ACAT  III  programs  have  the 
most  technically  detailed  operational  requirements,  but  do  gain  some  benefit  from 
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focused  oversight.  However,  ACAT  II  programs  endure  the  ‘worst  of  all  worlds’  when  it 
comes  to  T&E  burden.  It  appears  that  ACAT  II  CB  programs  are  managed  as  if  they 
were  ACAT  I-D  OSD.  DOT&E  oversight  programs,  while  also  having  an  ACAT  Ill-like 
burden  of  highly  technically-biased  operational  requirements.  Further  study  is  needed  to 
determine  whether  this  T&E  bias  applies  to  non-CB  programs,  or  is  unique  to  the  CBDP. 
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APPENDIX.  PROGRAM  FACT  SHEETS 


Future  Combat  Systems  (FCS) 

The  Army's  Future  Combat  Systems  (FCS)  network  allows  the  FCS  Family-of-Systems  (FoS)  to 
operate  as  a  cohesive  system-of-systems  where  the  whole  of  its  capabilities  is  greater  than  the 
sum  of  its  parts.  As  the  key  to  the  Army's  transformation,  the  network,  and  its  logistics  and 
Embedded  Training  <ET)  systems,  enable  the  Future  Force  to  employ  revolutionary  operational 
and  organizational  concepts.  The  network  enables  Soldiers  to  perceive,  comprehend,  shape,  and 
dominate  the  future  battlefield  at  unprecedented  levels  as  defined  by  the  FCS  Operational 
Requirements  Document  (ORD). 

The  FCS  network  consists  of  four  overarching  building  blocks.  System-of-Systems  Common 
Operating  Environment  (SOSCOE);  Battle  Command  <BC)  software;  communications  and 
computers  (CC);  and  intelligence,  reconnaissance  and  surveillance  (ISR)  systems.  The  four 
building  blocks  svnergistically  interact  enabling  the  Future  Force  to  see  first,  understand  first,  act 
first  and  finish  decisively. 

http,  //www.army.mil/fcs/overview.html  (Nov  2007) 


http.  //www4.army.mil'OCPA/uploads/large/FCSnetwork2004-07-23.jpg  (Nov  2007) 
http.  //www4.aimy.mil,OCPA/uploads'large'Mortar2004- 1 0- 1 9.jpg  (Nov  2007) 
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NBC  Reconnaissance  Vehicle  (NBCRV)  -  Stryker  Variant 


The  Nuclear.  Biological,  Chemical  Reconnaissance  Vehicle  (NBCRV)  provides  on  the  move  and 
remote  near-real-time  nuclear,  biological  and  chemical  detection  and  surveillance  to  supply 
battlefield  visualization  of  NBC  hazards. 

General, 

The  NBC  RV  provides  NBC  situational  awareness  to  increase  the  combat  power  of  the  SBCT. 
The  core  of  the  NBC’  RV  is  its  on-board  integrated  NBC  sensor  suite  and  integrated 
meteorological  system.  An  NBC  positive  overpressure  system  that  minimizes  cross- 
contamination  of  samples  and  detection  instruments,  provides  crew  protection,  and  allows 
extended  operations  at  MOPP  0. 

Operational  Capability. 

The  NBCRV  will  have  the  capability  to  detect  and  collect  chemical  and  biological  contamination 
in  its  immediate  environment  on  the  move  through  point  detection  (CBMS  and  JBPDS),  and  at  a 
distance  through  the  use  of  a  stand  off  detector  (JSLSCAD).  It  automatically  integrates 
contamination  information  from  detectors  with  input  from  on-board  navigation  and 
meteorological  systems  and  automatically  transmits  digital  NBC  warning  messages  through  the 
Maneuver  Control  System  to  warn  follow-on  forces.  Mission.  NBC  Reconnaissance.  Find, 
identity1,  map.  and  mark  NBC  contamination  on  the  non-linear  battlefield. 

•  32-57%  reduced  route  reconnaissance  mission  time  via  on-the-move  standoff  chemical 
agent  detection 

•  Common  NBC  technical  architecture 

•  Biological  Detection  Capability  (CBMS  and  JBPDS) 

Digitized  division/corps-fused  NBCRS  architecture 

Stryker  NBCRV  Technology 


http.  Z 'www.sbct.army.mil/  (Nov  2007),  http,  Z'www.jpeocbd.osd.mil’Recon/Products-CA- 
NBCStryker.html  (Nov  2007) 
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Stryker  Family  of  Vehicles 


Stryker™  is  a  family  of  eight-wheel  drive  combat  vehicles,  transportable  in  a  C-130  aircraft, 
being  built  for  the  US  Army  by  CiM  CiDLS,  a  joint  venture  set  up  by  General  Motors  Defense  of 
Canada  and  General  Dynamics  Land  Systems  Division  of  USA.  Stry  ker  is  based  on  the  CiM  LAV 
III  8x8  light-armored  vehicle,  in  service  since  early  2001.  The  LAV  III  is  itself  a  version  of  the 
Piranha  III  built  by  Mowag  of  Switzerland,  now  owned  by  General  Motors  Defense.  GM  Defense 
and  CiDLS  are  sharing  the  fabrication  and  final  assembly  of  the  vehicles  among  plants  at 
Anniston.  Alabama:  Lima.  Ohio;  and  London.  Ontario. 

The  Stryker  Brigade  Combat  Team  (SBCT)  combines  the  capacity  for  rapid  deployment  with 
survivability  and  tactical  mobility.  The  Stryker  vehicle  enables  the  team  to  maneuver  in  close  and 
urban  terrain,  provide  protection  in  open  terrain  and  transport  infantry  quickly  to  critical 
battlefield  positions. 

CiM  Defense  and  CiDLS  were  awarded  the  contract  for  the  US  Army's  Interim  Armored  Vehicle 
(IAV)  in  November  2000. 

The  eight-wheeled  Stryker  is  the  first  new  military  vehicle  to  enter  service  into  the  United  States 
Army  since  the  Abrams  tank  in  the  1980s. 

The  United  States  Army  first  deployed  14  Stryker  vehicles  as  part  of  its  forced  entry  package  for 
Millennium  Challenge  2002,  the  Joint  Forces  Command  field  experiment  and  demonstration  in 
July  and  August  2002.  The  Strykers  were  deployed  from  C-130  and  C-17  aircraft  during  the 
exercise.  Formal  brigade  certification  is  planned  for  May  2003. 

Variants, 

Stryker  variants  include  the  Infantry  Carrier  Vehicle  (ICV)  and  the  Mobile  Gun  System  (MGS). 
There  are  eight  configurations  of  the  ICV  including  Nuclear.  Biological.  Chemical 
Reconnaissance  Vehicle  (NBC  RV);  Anti-Tank  Guided  Missile  (ATGM);  Medical  Evacuation 
Vehicle  (MEV);  Mortar  Carrier  (MC);  Engineer  Squad  Vehicle  (ESV);  Commander’s  Vehicle 
(CV>;  Fire  Support  Vehicle  (FSV);  and  the  Reconnaissance  Vehicle  (RV).  They  have  parts 
commonality  and  self-recovery  abilities  and  are  equipped  with  a  central  tire  inflation  system. 


http.  / Avww.sbct.army.mil/  (Nov  2007) 
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Joint  Biological  Point  Detection  System  (JBPDS) 


Description. 

The  Joint  Biological  Point  Detection  System  (JBPDS)  Acquisition  Category  II  (AC AT  II) 
Sentinel  program  is  the  successor  to  the  Army  BIDS,  Navy  IBAD.  and  the  Air  Force  service 
specific  development  programs.  The  JBPDS  will  meet  Quad-service  requirements  as  outlined  in 
the  Joint  Operational  Requirements  Document  (JORD)  and  consist  of  complementary  trigger, 
sampler,  detector  and  identification  technologies  to  rapidly  and  automatically  detect  and  identify 
biological  threat  agents.  The  suite  will  be  capable  of  identifying  BW  agents  in  less  than  15 
minutes.  The  suite  will  be  capable  of  identifying,  as  a  minimum.  BW  agents  listed  in  Category  A 
of  the  International  Task  Force  (ITF)  6  Report,  dated  Feb  90.  The  detection  suite  will  be 
integrated  into  each  Service's  platform  (e.g.  BIDS,  surface  ships.  LNBCRS)  or  air  base  and  port 
to  provide  a  common  detection  capability  for  joint  interoperability  and  supportability.  The  JBPDS 
will  increase  the  number  of  agents  that  can  be  identified  by  the  BIDS  and  the  IBADS;  decrease 
detection  and  identification  time;  increase  detection  sensitivity;  provide  automated  knowledge- 
based  detection  and  identification;  and  provide  a  first  time  point  detection  capability  to  the  Air 
Force  and  Marine  Corps. 


Mission, 

To  detect,  identify,  sample,  collect  and 
communicate  the  presence  of  biological  warfare 
agents  to  enhance  the  surv  ivability  of  U.S.  Forces. 

Capabilities, 

-  Point  detection  capability  for  all  Services 

-  Increased  reliability  and  maintainability 

-  Identify  10  biological  warfare  agents 
simultaneously 

Improvements  over  the  BIDS  and  IBAD, 

-  Lower  probability  of  false  positive  identification 

-  Interface  with  JWARN  System  and  FBCB2 

-  Fully  automated  detection  and  identification 
capability 

User. 

U.S.  Army.  U.S.  Navy,  U.S.  Air  Force,  and  U.S. 
Marine  Corps 
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Joint  Chemical  Agent  Detector  (JCAD) 

Description. 

The  JCAD  will  be  a  combined  portable  monitoring  and  small  point  chemical  agent  detector  lor 
aircraft,  shipboard,  and  individual  soldier  applications.  This  hand-held,  pocket-sized  detector  is 
required  to  automatically  detect,  identify,  and  quantity-  chemical  agents  inside  the  aircraft  or  ship, 
providing  protection  for  the  individual  Soldier.  Sailor.  Airman,  or  Marine.  For  the  duration  of  the 
mission,  the  device  must  be  sufficiently  sensitive  to  warn  aircrews  before  accumulation  of  a  dose 
that  will  cause  miosis  or  more  severe  effects.  It  must  be  resistant  to  the  severe  interferent 
environment  on  a  naval  vessel  and  be  small  and  rugged  for  individual  use. 

Mission. 

Advanced  detection  and  warning,  identification  of  contamination  on  personnel  and  equipment, 
and  monitoring  for  the  presence  of  chemical  warfare  agent  contamination 

Capabilities, 

-  Instant  feedback  of  hazard  (mask  only  or  full  MOPP) 

-  Real-time  detection  of  nerve,  blister,  and  blood  agents 

-  Miosis-level  detection  capability  (with  pre-concentrator) 

-  Calculates  accumulated  dosage 

-  Stores  up  to  72  hours  of  detection  data 

-  Fully  compatible  with  Joint  Warning  and  Reporting  System  (JWARN) 

Users, 

U.S.  Air  Force  (Program  Lead),  U.S.  Army,  U.S.  Navy,  and  U.S.  Marine  Corps  Replaces  the 


Joint  Services  Lightweight  NBC  Reconnaissance  System  (JSLNBCRS) 


Description, 

The  JSLNBCRS  will  provide  point  and  standoff  intelligence  for  real-time  field  assessment  of 
NBC  hazards.  The  system  is  a  vehicle-mounted  suite  of  equipment  and  software  designed  to 
detect,  collect,  analyze,  mark,  and  disseminate  NBC  data.  Two  variants,  the  High  Mobility 
Multipurpose  Wheeled  Vehicle  and  the  Light  Armored  Vehicle,  will  house  the  same  equipment 
and  offer  on-the-move.  standoff  capability,  while  providing  an  air-transportable  system.  Timely 
provision  of  automated,  digital  information  meshed  with  meteorological  and  positioning 
information  will  provide  commanders  more  options  in  merging  NBC’  information  with  tactical, 
operational,  and  strategic  plans. 

Mission. 

Perform  NBC  Reconnaissance.  The  JSLNBCRS  will  detect,  identify,  mark,  collect,  correlate,  and 
disseminate  NBC'  hazard  and  toxic  industrial  chemical  information  over  an  automated  warning 
and  reporting  information  network. 

Capabilities, 

-  JSLSCAD  -  On-the-Move  near  real-time  chemical  vapor  detection 

-  CBMS  II-  Chemical  Detection 

-  JBPDS-  Biological  Detection 

-  AC  ADA  -  Chemical  warfare  agent  detection  (nerve  and  blister  agents) 

-  ADM  300A  -  AN'VDR-2  -  Measures  radioactivity  for  either  vehicle-mounted  or  hand-held 
operations 

-  ICAM/CAM  II  -  Hand-held  monitoring  of  chemical  agents 

-  JWARN  to  analyze  and  report  NBC  hazards 
Improvements, 

-  On-the-move  standoff  chemical  detection 

-  Biological  detection  and  sampling 

-  On-the-move  meteorological  system 

-  Significantly  less  impact  on  strategic  and  intra-theater  lift  resources 

-  Collective  protection  for  crew  against  known  CBW  agents. 

-  Generation  of  automated  NBC  warning  messages. 


Users, 

U.S.  Marine  Corps  and  U.S.  Ait  Force 


http.  / Avww.jpeocbd.osd.mil/Recon/Products-CA-JSLNBCRS.html  (Nov  2007) 
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M22/M88  Automatic  Chemical  Agent  Detection  Alarm  (ACADA) 


Description. 

The  M22  Automatic  Chemical  Agent  Alarm  (ACADA)  is  an  automatic  chemical  agent  alarm 
system  capable  of  detecting,  warning  and  identifying  standard  blister  and  nerve  agents 
simultaneously.  The  M22  is  man-portable,  operates  independently  after  system  start-up.  provides 
an  audible  and  visual  alarm  and  provides  communication  interface  to  support  battlefield 
automation  systems. 

Mission. 

Provide  detection  and  warning  for  nerve  and  blister  agents. 

Capabilities, 

-  Area  warning 

-  Collective  Protection  Equipment  (CPE)  monitoring 

-  Operation  on  and  in  vehicles 

-  Compatible  with  MIC  AD 

-  Prov  ides  simultaneous  detection  and  warning  of  nerve  and  blister  agents 

-  Significantly  more  sensitive  than  MSA  I 

-  Operates  in  a  collective  protection  environment 

-  Much  less  responsive  to  interferences 

-  Able  to  operate  in  and  on  vehicles 


Users, 

U.S.  Army,  U.S.  Navy.  U.S.  Air  Force  and  U.S.  Marine  Corps 


http,  /  Avww.jpeocbd.osd.mil/Point/Products-CA-ACADA.html  (Nov  2007) 
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Analytical  Laboratory  System  (ALS) 


Description,  The  Analytical  Laboratory  System  (ALS)  is  a  CM 30  air  transportable  system 
thatuses  commercial-off-the-shelf  (COTS)  equipment  that  can  analyze  Chemical  Warfare  (CW) 
agents.  Toxic  Industrial  Materials  (TIM),  Toxic  Industrial  Chemicals  (TIC)  and  Biological 
Warfare  (BW)  agents.  The  ALS  has  the  capability  of  establishing  communications  through  the 
Unified  Command  Suite  (UCS)  to  local,  state,  federal  laboratories  and  other  Agencies  for 
confirmatory  analysis  of  supect  agent. 

Mission,  The  ALS  provides  the  capability  to  the  Civil  Support  Teams  to  conduct  presumptive 
analysis  of  unknown  or  potential  agents  at  an  incident  site  and  transmit  that  information 
electronically  via  the  UCS  in  support  of  the  First  Responder  Incident  Commander. 

Capabilities, 

Detects  and  identifies  GB,  GD  and  HD 

Self  -  contained  and  fully  functional  and  reliable  in  the  range  of  extreme  climates  encountered  in 
the  United  States  and  its  territories 

Provides  results  from  screening  of  TIC/TIM/CW  samples  within  30  minutes 
Provides  results  from  screening  of  BW  samples  within  45  minutes 

User,  National  Guard  Bureau,  Weapons  of  Mass  Destruction  -  Civil  Support  Teams. 


http. /Avww.jpeoebd.osd.mil’pagemanager.asp'.’pg  2&sub  19  (Nov  2007) 


86 


Biological  Integrated  Detection  System  (BIDS) 

Description. 

The  Biological  Integrated  Detection  System  (BIDS)  is  an  early  warning  and  identification 
capability  in  response  to  a  large  area  (theater)  Biological  Warfare  (BW)  attack.  The  system  is  a 
detection  suite  installed  in  a  shelter  that  is  mounted  on  a  dedicated  vehicle  with  trailer  mounted 
generator  power  supply.  Other  BIDS  elements  include  collective  protection,  environmental 
control,  sample  storage.  GPS,  MET.  and  radios.  The  BIDS  pre-planned  product  improvement 
BIDS  (P31)  system  is  equipped  with  a  detection  suite  to  include  high  volume  samplers,  a 
fluorescent  particle  counter/sizer,  a  flow  cytometer,  a  chemical/biological  mass  spectrometer  and 
an  antibody-based  biological  detector.  The  shelter  may  be  removed  from  the  vehicle  for  fixed  site 
application.  The  BIDS  program  was  conducted  in  three  phases.  Phase  1  was  the  noil- 
developmental  item  (ND1)  BIDS.  Pliase  II  was  the  P3I.  which  provided  technology  insertion  to 
upgrade  the  NDI  core  configuration  from  a  four-agent  to  an  eight-agent  identification  capability 
with  improved  generic  detection.  The  objective  system  (Phase  II)  transitioned  to  JBPDS. 

Mission, 

Detect  and  identify  large-area  Biological  Warfare  (BW)  agent  attacks,  provide  a  basis  for  large- 
area  protection  and  warning. 

Capabilities, 

-  BIDS  NDI 

-  Detect/Identify  25  ACPLA/30Min 

-  Manual  operation 

-  BIDS  P3I 

-  Detect/Identify  15  ACPLA/20Min 

-  Semi-automatic  operation 

-  BIDS  Obj  System 

-  JBPDS  Bio  Suite 

Improvements  over  prior  systems. 

-  First  Army  biological  detection  capability 

-  Basis  for  rapid  detect-to-treat  decisions 

-  Provide  improved  situational  awareness 

User. 

U.S.  Army 
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Chemical  and  Biological  Protective  Shelter  (CBPS) 


Description, 

The  Chemical  and  Biological  Protective  Shelter  (CBPS)  provides  a  highly  mobile,  contamination 
free,  environmentally  controlled  work  area  for  forward  deployed  medical  treatment  units.  The 
CBPS  consists  of  a  dedicated  Ml  1 13  HMMWV  vehicle,  a  Lightweight  Multipurpose  Shelter 
(LMS)  mounted  on  the  back  of  the  HMMWV,  a  300  ft*  semi-cylindrical  airbeam-supported  soft 
shelter,  and  a  towed  High  Mobility  Trailer  with  IOk\V  Tactical  Quiet  Generator  (TQG)  for 
auxiliary  power.  Four  soldiers  operate  the  CBPS;  during  transport,  two  ride  in  the  HMMWV  and 
two  in  the  LMS.  Soldier's  gear  and  required  medical  equipment  will  be  stored  in  the  LMS  or  on 
the  trailer.  The  inflatable  shelter  is  rolled  and  carried  on  the  rear  of  the  LMS  during  transport.  All 
power  required  to  support  operations  of  the  sys-tem  is  provided  by  the  HMMWV  engine  via  an 
underhood  hydraulic  pump,  or  from  the  auxiliary  lOkW  TQG.  A  hydraulically-powered 
Environmental  Support  System  mounted  on  the  front  of  the  LMS  provides  heating,  cooling, 
airbeam  inflation.  CB  filtration,  and  ventilation  air. 

Mission. 

CBPS  will  be  fielded  to  Treatment  Squads.  Forward  Surgical  Teams,  and  Medical  Companies 
and  will  allow  medical  personnel  to  work  without  encumbrances  of  protective  clothing  in  a  CB 
contaminated  environment. 

User,  Full  Materiel  Release  achieved  and  fielded  to  U.S.  Army  and  U.S.  Marine  Corps  during 

Operation  Iraqi  Freedom 

Capabilities, 

•  Highly  Mobile  self-contained  system 

•  (.'BPS  can  be  fully  operational  in  less  than  20  minutes 

•  Personnel  work  inside  without  encumbrances  of  protective  gear 

•  Provides  a  clean  environment  for  medical  care,  to  include  a  desert  environment 


o  ti 

•  Environmentally  controlled  sustained  operations  25  F  to  120  F 


http.  //www. ipeocbd.osd.mil/documents/CBPS_llandout2.pdf (Nov  2007) 


M93A1  Fox  Nuclear,  Biological,  Chemical  Reconnaissance  System 

(NBCRS) 


Description. 

The  M93A1  FOX  NBCRS  is  a  dedicated  system  of  nuclear  and  chemical  sampling,  detection,  and 
warning  equipment,  and  biological  sampling  equipment  integrated  into  a  high  speed,  high 
mobility,  six  wheel  armored  vehicle.  It  is  capable  of  performing  nuclear,  biological,  and  chemical 
(NBC)  reconnaissance  on  primary,  secondary,  or  cross-country  routes  throughout  the  battlefield. 
The  M93AI  is  the  Block  I  improvement  of  the  M93  FOX  NBCRS  Interim  System  to  meet  all  of 
the  requirements  of  the  approved  required  operational  characteristics  (ROC),  and  reduce  the  crew 
size  to  three.  It  is  capable  of  detecting  chemical  contamination  in  its  immediate  environment 
through  point  detection,  and  at  a  distance  through  the  use  of  a  standoff  detector  (M21  Chemical 
Agent  Detector  Alarm).  It  automatically  integrates  contamination  information  from  detectors  with 
input  from  on-board  navigation  and  meteorological  systems  and  automatically  transmits  digital 
NBC  warning  messages  through  the  Maneuver  Control  System  to  warn  follow-on  forces. 

Mission. 

NBC  Reconnaissance.  Find,  identify,  map.  and  mark  NBC  contamination  on  the  battlefield. 
Capabilities, 

-  Standoff  vapor  detection 

-  Reduced  crew  size 

-  Digitized  communications 

-  On-board  MICRO-MET,  GPS,  SINCGARS 

-  Automatic  Alarm  Reporting  (MICAD) 

Future  Improvements. 

-  FOX  Recapitalization  -  Zero  hours/Zero  Miles.  Enhanced  Sensor  Suite.  Improved  Survivability 
and  Lethality 

-  Significantly  reduced  operational,  sustainment  and  life  cycle  costs 


Users, 

U.S.  Army,  U.S.  Marine  Corps 


http.  .'/www. jpeoclxl.osd.mil 'Recon  Products-CA-FOXBIockl.html  ( Nov  2007) 
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Joint  Biological  Agent  Identification  and  Diagnostic  System  (JBAIDS) 


Description,  The  Joint  Biological  Agent  Identification  and  Diagnostic  System  (JBAIDS)  is  an 
integrated  system  for  rapid  identification  and  diagnostic  confirmation  of  biological  agent 
exposure  or  infection.  Based  on  commercial  technology,  JBAIDS  is  man-portable,  reusable,  and 
will  be  capable  of  the  simultaneous  identification  of  multiple  Biological  Warfare  agents  (BWA) 
and  other  pathogens  of  operational  concern.  The  system  consists  of  the  hardware  platform  to 
perform  sample  analysis,  a  laptop  computer  for  readout  display,  and  assay  reagent  test  kits. 

Mission, 

Provide  rapid  positive  identification  and  diagnostic  confirmation  of  BWA  and  other  pathogens  of 
operational  concern. 

Capabilities, 

•  Single  DoD  accepted  platform  for  both  identification  and  diagnostic  confirmation  of 
biological  agents 

•  Operation  in  fixed  medical  laboratories  and  deployed  medical  units 

•  Operates  as  a  stand-alone  system;  future  development  Blocks  II  and  111  to  be 
interoperable  with  Theatre  Medical  Information  Program  (TMIP) 

Improvements  over  existing  technology, 

•  Provides  simultaneous  identification  of  multiple  biological  agents 

•  Rapid  identification 
User, 

U.S.  Army,  U.S.  Navy,  U.S.  Air  Force,  and  U.S.  Marine  Corps. 


http,//www.jpeocbd.osd.mil/page_manager.asp?pg  2&sub  54  (Nov  2007) 
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Joint  Biological  Standoff  Detection  System  (JBSDS) 


Description, 

The  JBSDS  is  in  development  and  is  the  first  joint  biological  standoff  detection  program.  The 
JBSDS  will  be  a  standoff  early  warning  biological  detection  (BD)  system.  The  system  will  be 
capable  of  providing  near  real  time,  detection  of  biological  attacks/incidents  and  standoff  early 
warning  detection/warning  of  biological  w  arfare  (BW)  agents  at  fixed  sites  or  when  mounted  on 
multiple  platforms,  including  NBC  reconnaissance  platforms.  It  will  be  capable  of  providing 
standoff  detection,  ranging,  tracking,  discrimination  (manmade  vs  natural  occurring  aerosol)  and 
generic  detection  (bio  vs  non-bio)  of  large  area  BW  aerosol  clouds  for  advanced  warning, 
reporting,  and  protection. 

Mission. 

To  provide  a  biological  detection  network  capable  of  near  real  time  detection  and  warning, 
theater  wide,  to  limit  the  effects  of  biological  agent  hazards  at  the  tactical  and  operational  levels 
of  war. 

Capabilities, 

-  Provides  integrated,  stand-off  detection  near  real  time  for  Bio-  Detect  and  track  aerosol  clouds 
out  to  15  km 

-  Operate  at  fixed  site  or  stationary  mode  from  mobile  platforms 

-  Discriminate  Bio  particles  from  non  Bio  particles  in  aerosol  clouds  out  to  3km 

-  Operationally  skin  and  eye  safe 
Improvements  over  previous  systems, 

-  JBSDS  will  provide  early  w  arning  BWA  standoff  for  all  Services.  Currently  fielded  biological 
detection  systems  do  not. 

-  JBSDS  will  discriminate  between  natural  and  man-made  aerosols. 

-  JBSDS  w  ill  not  require  aviation  assets  for  employment.  Existing  LRBDS  does. 

User, 

U.S.  Army,  U.S.  Navy,  U.S.  Air  Force,  and  U.S.  Marine  Corps 


http.  /Avww.jpeocbd.osd.mil/page  manager.asp'.’pg  7&sub  1 9  (Nov  2007) 
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Joint  Protective  Aircrew  Ensemble  (JPACE) 


Description, 

The  Joint  Protective  Aircrew  Ensemble  (JPACE)  is  a  Chemical  Biological  Protective  garment 
that  is  fire  resistant  and  provides  protection  from  Chemical/Biological  (CB)  warfare  agents  and 
radiological  particles.  JPACE  is  intended  for  use  by  all  aviators  and  aircrew  for  fixed  wing 
ejection,  fixed  wing  non-ejection,  and  rotary  wing  personnel. 

Mission. 

Provide  below-the-neck  chemical/biological  protection  for  aviators  and  aircrew  personnel. 
Capabilities, 

♦  Chemical  protection  from  all  liquid,  particle,  vapor  and  aerosol  CB  agents 

♦  Provides  protection  to  enhance  combat  capabilities  in  a  CB  environment 

♦  Compatible  with  all  services  aviation  ALSS/ALSE.  legacy  Chein'Bio  Aviation  mask  systems. 

all  developmental  masks,  JSLIST  Glove  Block  Upgrades,  foot  wear  solutions,  and  the 
Army's  Air  Warrior  system. 

Improvements  over  the  current  garments. 

Replaces,  Navy  Mkl,  CPU-CWU-27P; 

Army  ABDU-BDO,  CPU-ABDU;  Air 
Force  CWU-66/P  Increased  CWBW  agent 
protection 

♦  Increased  flame  protection 

♦  Longer  post  exposure  protection 

♦  Decreased  heat  stress 

Repair  kits  for  both  the  outer  shell  and 
the  line  for  field  expedient  and  repair  for 
reuse. 

Laundering  cycles,  shelf/service  life  is 
increased. 

User. 

U.S.  Army,  U.S.  Navy,  U.S.  Air  Force, 

U.S.  Marine  Corps.  U.S.  Coast  Guard 
and  U.S.  Special  Operations  Command. 
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XM50/51  Joint  Service  General  Purpose  Mask  (JSGPM) 


The  JSGPM  will  be  a  lightweight,  protective  mask  system  (consisting  of  mask,  carrier  and 
accessories)  incorporating  state  of  the  an  technology  to  protect  U.S.  forces  from  anticipated 
threats.  The  mask  components  will  be  optimized  to  minimize  impact  on  the  wearer’s  performance 
and  to  maximize  its  ability  to  interface  with  future  Service  equipment  and  protective  clothing. 

Mission, 

Provide  face,  eye.  and  respiratory  protection  from  battlefield  concentrations  of  Chemical  and 
Biological  <CB)  agents,  toxins,  toxic  industrial  materials,  and  radioactive  particulate  matter. 

User. 

All  Services  -  Replaces  the  M40/42  and  MCU-2/P  Series  Masks,  and  the  M45  in  the  Land 
Warrior  Program. 

Target  Capabilities.  -  Provide  24  hours  of  above  the  neck  protection  from  CB  agents,  radioactive 
particles,  and  toxic  industrial  materials-  Overall  field  of  view  80% 

-  Inhalation  resistance  30  mm  water 

-  Compatible  with  current  and  co-developmentai  CB  garments 


Target  Improvements  over  the  M40/42,  MCU-2  P.  and  M45, 

-  Reduced  weight  and  bulk 

-  Lower  breathing  resistance 

-  Improved  equipment  compatibility 

-  Improved  and  increased  periods  of  protection 


http,  //www .jpcocbd.osd.mil/pagc_managcr.asp?pg=2&sub=40  (Nov  2007) 
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Joint  Block  I  Glove  Upgrade  Program  (JB2GU) 


Description, 

The  JB2GU  is  a  system  that  will  provide  protection  against  chemical/biological  (CB)  warfare 
agents.  It  is  a  component  of  the  JSLIST  ensemble  and  will  improve  upon  the  JB1GU  effort  by 
offering  greater  durability  that  will  satisfy  a  broader  set  of  known  requirements,  specifically, 
JSLIST  ground  and  shipboard  use  and  aviation.  JB2GU  will  be  used  with  the  JSLIST  and  JPACE 
ensemble  and  chemical  protective  mask. 

Target  Capabilities, 

The  JB2GU  will  provide  hand  protection  from  liquid,  vapor,  and  aerosol  Chemical/Biological 
(C'B)  hazards.  It  w  ill  provide  enhanced  tactility,  dexterity,  and  comfort  over  existing  systems  and 
can  be  wont  in  all  climates.  The  glove  will  offer  24  hours  of  protection  in  a  contaminated 
environment  and  is  durable  up  to  30  days. 

How  JB2GU  is  Used, 

The  warfighter  will  use  the  JB2GU  glove  along  with  the  JSLIST  and  JPACE  ensembles,  footwear 
and  masks  to  create  a  state-of-the-art  chemical/biological  protective  ensemble  that  provides 
complete  percutaneous  and  respiratory  protection  against  chemical  and  biological  warfare  (CBW) 
agents. 

User. 

U.S.  Navy,  U.S.  Air  Force,  U.S.  Army,  and  U.S.  Marine  Corps. 


http.  /Av ww.jpeocbd.osd.mil/page_manager.asp7pg  2&sub  36  (Nov  2007) 
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Joint  Services  Lightweight  Standoff  Chemical  Agent  Detector 
(JSLSCAD) 

Description, 

The  Joint  Services  Lightweight  Standoff  Chemical  Agent  Detector  (JSLSCAD)  is  a  state-of-the- 
art  detection  system  designed  to  provide  U.S.  Forces  with  enhanced  capability  in  detecting 
chemical  warfare  agents.  It  is  a  lightweight,  passive,  and  fully  automatic  detection  system  that 
scans  the  surrounding  atmosphere  for  chemical  warfare  agent  vapors.  It  furnishes  on-the-move, 
360”  coverage  from  a  variety  of  tactical  and  reconnaissance  platforms  at  distances  up  to  5 
kilometers.  It  is  a  second-generation  system  that  significantly  improves  on  the  capabilities  of  the 
currently  fielded  M21  Remote  Sensing  Chemical  Agent  Alarm  (RSCAAL).  The  JSLSCAD  will 
provide  war  fighters  with  enhanced  early  warning  to  avoid  chemically  contaminated  battlespace. 
When  avoidance  is  not  possible.  The  JSLSCAD  will  give  personnel  extra  time  to  don  MOPP 
gear. 

Mission, 

Provide  standoff  detection  and  warning  for  nerve,  blister,  and  blood  agent  vapor  clouds. 
Capabilities, 

-  Detects,  identifies  and  reports  nerve,  blister,  and  blood  agent  vapor  clouds 

-  Mounts  on  ground,  air  and  sea  platforms 

-  Provides  360p  by  60°  on-the-move  coverage 

-  Provides  up  to  5  km  detection  range 

-  No  operator  required 

-  Automatic  warning  and  reporting  through  JWARN 
Users, 

U.S.  Army,  U.S.  Marine  Corps,  Future,  U.S.  Air  Force,  U.S.  Navy 

Scanner 
Module 


Sensor 

Electronics 

Module 


Power 
Adapter 

http.  /Avw’W'. jpeocbd.osd.miLStandoffProduets-C.A-JSLSC.AD.html  (Nov  2007) 
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JSFDS  Joint  Services  Family  of  Decontamination  Systems 
(Includes  JSTDS-SSI 


Description,  Joint  Service  Family  of  Decontamination  Systems  (JSFDS)  is  designated  to 
incorporate  across  the  DoD  a  family  of  decontamination  systems  that  will  provide  personnel, 
equipment  and  area  decontamination  of  all  chemical  and  biological  warfare  agents  to  a  level  safe 
for  return  to  unrestricted  operational  use. 

Mission,  To  develop  chemical  and  biological  decontamination/application  systems  for 
equipment,  wounded  and  non-wounded  personnel. 

User,  U.S.  Army,  U.S.  Navy,  U.S.  Air  Force,  and  U.S.  Marine  Corps 
M100NSN,  4230-01-466-9095  Bracket  NSN.  5340-01 -466-592X 
Capabilities, 

•  Decon  of  fixed  facilities,  polls  of  entry,  airfields 

•  Dispensing  of  the  entire  family  of  decontaminants  regardless  of  form 

•  Personnel  skin  decontamination  capability  for  use  on  casualties  -  with  and  without 
open  wound. 

•  Improvements  over  current  capabilities 

•  Provides  a  common  capability  to  decontaminate  facilities,  structures,  etc. 

•  Introduction  of  skin  decontamination  treatment 
Unification  of  dispensing  methods 

NOTE.  Requirements  for  the  JSTDS-SS  ure  found  in  the  Joint  Service  Family  of 
Decontamination  Systems  (JSFDS)  Joint  Service  Transport  a  hie  Decontamination  System 
(JSTDS)  Milestone  (MS)  B  ORD,  Acquisition  Category  (AC AT)  III.  Increment  /  dated  19 
February  2004.  approved  I  March  04.  (  Ref  p.  I.  JSTDS-SS  TEMP) 
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Light  Vehicle  Obscuration  Smoke  System  (LVOSS) 


Description, 

The  LVOSS  is  a  self-defense  smoke  and  riot  control  device  externally  mounted  on  the  host 
vehicle.  It  consists  of  grenades,  M7  Dischargers  to  lire  the  grenades,  and  installation  kits  to 
mount  the  dischargers  on  vehicles. 

Mission. 

Provide  smoke  protection  and  riot  control  for  light  vehicles.  The  LVOSS  uses  M90  smoke 
grenades  to  counter  threat  weapon  systems  operating  in  the  v  isual  and  near-infrared  portion  of  the 
electromagnetic  spectrum.  The  system  can  also  bo  used  to  deliver  anti-riot  grenades  (L96A1. 
L97A1,  M98,  and  M99). 

Capabilities, 

-  Standardized  discharger  design  for  mounting  on  any  family  of  light  vehicle 

-  M304  Installation  Kit  installs  one  M7  Dischargers  on  M966  HMMWVs 

-  M305  Installation  Kit  installs  four  M7  Dischargers  on  M  1025/M  1026  HMMWVs 

-  M3 10  Installation  Kit  installs  four  M7  Dischargers  on  Mil  14  HMMWVs 

-  600  coverage  per  discltarger 

Improvements  over  previous  systems, 

-  Provides  smoke  protection  to  light  vehicles  (This  capability  previously  did  not  exist  for  light 
vehicles) 

-  Uses  grenades  that  are  less  toxic  &  are  safer  than  those  used  on  armored  vehicles 

Users. 

U.S.  Army 


V 


http,  .''www.jpcocbd.osd.mil.  Recon'Products-CA-L VOSS.html  (Nov  2007) 
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M45  Mask 


Description, 

The  M45  Mask  is  used  by  all  Army  aircrew  members  except  AH-64  (Apache)  helicopter  pilots  in 
the  conduct  of  aviation  missions  anywhere  in  a  CB  environment.  The  mask  consists  of  close- 
fitting  eye  lenses,  front  and  side  voicemitters  for  face-to-face  and  telephone  communication,  a 
low  profile  canister  interoperability  hose  assembly  to  allow  both  hose  and  face  mounted 
configurations,  a  rubber  facepiece  with  an  in-turned  peripheral  seal,  a  second  skin,  and  a  hood. 
The  mask  provides  the  required  CB  protection  without  the  aid  of  forced  ventilation  air.  The  M45 
Mask  supports  the  Land  Warrior  program  and  senes  as  the  mask  for  personnel  who  cannot  be 
fitted  with  the  standard  M40A1.  M42A2.  or  MCU-2A  P  protective  masks. 

Mission, 

Provide  aircrew  and  hard-to-fit  personnel  with  protection  against  all  known  chemical  and 
biological  threat  agents  and  radiological  particulates. 

User, 

U.S.  Army,  U.S.  Navy,  U.S.  Air  Force,  and  U.S.  Marine  Corps. 

Target  Capabilities, 

Compatible  with  night  vision  devices  and  aircraft  sighting  systems 

-  Microphone  pass-through  for  aircraft  communication 

-  Interchangeable  nosecups 

-  Drink  tube  for  liquid  nutrients 

Improvements  over  the  M24  and  M43  Type  II  Aircraft  Masks, 

-  Eliminates  blower  and  battery  requirements 

-  Remains  flexible  at  extreme  temperatures 

-  Greater  facial  seal  contact  area 

-  Removable  hose  can  be  installed  on  either  side  of  the  mask 


http,  //www.jpeocbd.osd.miLpage_manager.asp?pg  2&sub  45  (Nov  2007) 


\156  Motorized  Smoke  Obscurant  System  "The  Coyote" 


Description. 

The  MS6  Smoke  Generating  System  is  the  U.S.  Army's  first  visual  and  infrared  (IR)  large  area 
smoke  generating  system.  The  system  consists  of  visual,  infrared,  and  power  modules  mounted 
on  the  back  of  an  Ml  1 13  High  Mobility  Multipurpose  Wheeled  Vehicle.  Fog  oil  is  pumped  into 
the  exhaust  gas  of  a  turbine  engine  to  produce  visual  obscuration  and  graphite  pellets  are 
pulverized  to  5  micron  particles  and  disseminated  through  a  separate  ejector  to  produce  infrared 
obscuration. 

Mission. 

Defeat  all  threat  reconnaissance,  surveillance,  and  target  acquisition  (RSTA)  systems  to  deny  the 
enemy  information,  protect  forces  and  dominate  the  maneuver  battle. 

Capabilities, 

-  Simultaneous  visual/IR  obscuration 

-  Visual  -  90  minutes  (fog  oil) 

-  IR  -  30  minutes  (graphite) 

-  AN/V AS-5A( V)2  Driver's  Vision  Enhancer 

-  C-130  Transportable,  Air  Droppable,  01-47  Sling-Load 

-  SINCGARS  radio  and  AN/VIC-3  intercom 

Improvements  over  legacy  systems. 

-  (ireater  mobility 

-  IR  capability 

-  Ease  of  operation 

Users, 

U.S.  Army 
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Discharger,  Grenade,  Smoke,  Countermeasure,  M6 


Description, 

The  \16  is  a  4-tube  smoke  grenade  discharger  that  enables  combat  vehicles  to  conceal  themselves 
from  threat  surveillance,  target  acquisition,  and  weapon  guidance  systems.  It  is  a  2  X  2  tube 
design  weighing  9.8  lbs. 

Mission, 

Provides  smoke  protection  to  armored  vehicles. 

Capabilities, 

-  Four  tubes  are  independently  addressable 

-  Mounts  onto  all  armored  platforms 

-  Capable  of  firing  ail  standard  66mm  smoke  grenades 
-2x2  design 

-  Tubes  set  14.5  degrees  apart;  8"x7"x9"  long;  9.8  pounds 

-  Cast  aluminum 

Improvements  Over  Previous  Grenades, 

-  Individual  fire  control  capability 

-  Two  loaded  salvos 

Users, 

U.S.  Army 


http.  //www.jpeocbd.osd.mil/Recon/Products-CA-M6.html  (Nov  2007) 
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Shipboard  Collective  Protection  System  Backlit  Program 


[This  Information  is  for  a  more  recent  Shipboard  CPS  system,  as  no  digital  data  sheet  was  found 
on  the  older  ( evaluated >  CPS  version  for  ships  j 

Purpose, 

The  shipboard  Collective  Protection  System  (CPS)  Backlit  Program  was  created  to  provide 
additional  collective  protection  capabilities  to  existing  amphibious  class  ships  allowing  personnel 
to  perform  mission  critical  operations  is  a  Chemical.  Biological,  and  Radiological  (CBR) 
environment. 

Major  Components, 

M9X  Gas-particulate  filter  sets,  pre-filter  bags,  filter  housings,  van-axial  fans,  fan  rooms,  air 
locks,  pressure  gages,  zone  pressure  relief  values,  zone  alarms  and  control  panels  and  a 
decontamination  station. 

Mission. 

The  program  installs  additional  CPS  toxic  free  zones  in  critical  areas  including  medical, 
command  and  control,  rest  and  relief,  and  casualty  collecting  areas.  Personnel  working  in  these 
protected  areas  do  not  have  to  wear  individual  protective  equipment  (1PE)  during  or  after  a  CBR 
attack. 

Capabilities, 

♦  Additional  toxic  free  zones  that  allow  ships 
personnel  to  operate  in  a  CBR  environment 

♦  Full-time  operation 

♦  3+  year  filter  life 

♦  LH.A'LHD  class  ships  now  capable  of 
receiving  and  treating  contaminated  casualties 
in  a  collectively  protected  environment 

User, 

U.S.  Navy 


http.  /  Avww.jpeocbd.osd.mil/documents/CP_CPS_Backfit.pdf  (Nov  2007) 
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